Welcome to The Service Works: Building AI-Powered Applications with Elixir

There's something fascinating happening at the intersection of functional programming and artificial intelligence. While the tech world rushes to integrate AI into everything, most developers are reaching for the familiar tools: Python for the AI integration, JavaScript for the frontend, and whatever they know best for the backend. But what if there's a better way?

What if we could build AI-powered applications that are not just smart, but genuinely reliable? Systems that handle AI workloads gracefully, scale effortlessly, and—here's the kicker—actually stay running when things go wrong?

That's exactly what you get when you combine Elixir's battle-tested concurrency model with the capabilities of modern AI. And it's what this blog is mostly about.

Why Elixir for AI?

At first glance, Elixir might seem like an odd choice for AI development. After all, most machine learning happens in Python, and most AI tutorials assume you're working in that ecosystem. But here's what those tutorials don't tell you: building production AI applications involves a lot more than just calling a model.

You need to handle concurrent requests from hundreds of users. You need to manage real-time data streams. You need systems that can gracefully degrade when external AI services are down. You need to orchestrate complex workflows that might involve multiple AI models, data transformations, and business logic. And you need all of this to work reliably, day after day, without constant firefighting.

This is where Elixir shines. Built on the Erlang Virtual Machine (BEAM), Elixir was designed from the ground up for exactly these kinds of challenges. The telecoms systems that Erlang originally powered had to handle millions of concurrent connections with "five nines" reliability. Those same principles make Elixir perfect for AI applications that need to scale and stay running.

Elixir really excels in the orchestration layer around AI. Think about some of the things a production AI system actually needs:

• MCP (Multi-User Chat Protocol) servers: There are multiple libraries for writing MCP servers in Elixir (one of the most mature is Hermes). Using Elixir for MCP servers is a natural fit because Elixir’s strengths align well with the requirements of real-time, concurrent communication systems. Elixir remote MCP servers can also be easily deployed to services like Fly.io
• Data Ingestion & Preprocessing: Elixir’s concurrent, fault-tolerant architecture is well-suited for building pipelines that collect, clean, validate, and transform large datasets in real-time, preparing them for AI/ML models. Tools like Broadway and Flow make it easy to build resilient and scalable data workflows.
• MLOps & System Orchestration: Elixir’s strengths in distributed systems and orchestration make it ideal for building parts of the AI stack including managing retraining pipelines and scheduling jobs (for example using Oban job processing).
• Python Integration: Using various integration techniques we can bridge the gap between Elixir’s reliability and Python’s mature AI ecosystem. One of the newest ways of doing this is embedding Python within Elixir using Pythonx

The above list is just a few examples of the work that is being done by the Elixir community in the area of AI - some others include: Nx, Axon, Langchain and Bumblebee.

AI and Reliability

While much of the AI world is caught up in a kind of "move fast and break things" mentality, production AI systems require a different approach. When your AI-powered application is handling customer data, financial transactions, or healthcare information, "breaking things" isn't an option.

Elixir's "let it crash" philosophy might sound contradictory, but it's actually the perfect complement to AI development. AI models are inherently probabilistic—they will occasionally produce unexpected outputs, external APIs will fail, and data pipelines will encounter edge cases. Instead of trying to prevent all failures (which is likely impossible), Elixir helps you build systems that fail gracefully and recover automatically.

This approach helps reduce the developer anxiety that comes with deploying AI systems. Instead of lying awake wondering if your chatbot is hallucinating inappropriate responses to customers, you build systems with proper supervision, circuit breakers, and fallback mechanisms.

Using AI to Build AI Applications

It also turns out that Elixir is increasingly a great environment for using AI to build applications (which themselves may include AI integration). When developing with Elixir, AI can be successfully used to generate code, test cases and associated documentation.

Recent advancements include using the Tidewave tools which can connect a live Elixir system to AI tools, resulting in much improved Agentic AI as well as the Phoenix.new remote AI development tools from Fly which can be used to create Phoenix Apps without having to install software on your local machine.

What You'll Find Here

This blog will hopefully be detailed and practical. The posts will often be long-form in nature, consisting of mostly text and multiple code samples.

Below are some of the topics that will eventually be covered. Many of the examples and libraries from the discussions above will also be included.

Elixir Fundamentals: Including syntax, features, patterns and anti-patterns. We'll explore OTP (Open Telecom Platform) design patterns, supervision and reliability strategies, and how to structure applications that can handle real-world complexity.

Testing Strategies: Beyond basic unit tests, we'll look at property-based testing with StreamData or PropCheck, integration testing with Phoenix, load testing and benchmarking with tools like Benchee.

Production Readiness: Monitoring with Telemetry, observability with tools like LiveDashboard, deployment strategies that minimise risk, and how to debug issues when they inevitably arise.

AI Code Generation: Practical examples of using AI tools to enhance development workflows, generate better tests, and build better applications.

AI Integration: Detailed examples of using Elixir to create AI powered applications, tools and assistants.

Whether you're an experienced Elixir developer curious about AI, an AI enthusiast looking for better infrastructure tools, or a developer who wants to build AI systems that actually work reliably in production, this blog is for you.