I run multiple AI agents locally on this Mac, with MLX. No cloud, no API keys. The hardware does the work. Virtually no cost.
Over the past year, AI agents have gone from research prototypes to everyday productivity tools. Before I talk about the agent, the thing I had was a chat box.
What a chat box leaves to me
The chat I already knew works like this. I send a prompt to the language model. The model sends a response back. If I need to act on that response, run a command, check a file, or fix an error, that is on me. The model stopped at the text.
Now I talk to an agent. The agent talks to the model to decide what to do. Then it calls tools to actually do it: running commands, reading files, hitting APIs. It observes the results and goes back to the model to figure out the next step.
User to agent, agent to model, agent to tools. This is the agentic loop, and it keeps cycling until the task is done.
The loop runs on the machine
On Apple Silicon the entire loop can run locally. The data stays on the machine. The model is available anywhere, at any time, because it is already on the Mac in front of me. There are no usage costs.
MLX on the left, OpenCode on the right
I have an agent running locally on this Mac. On the left, MLX is running the model. On the right, OpenCode is the agent I talk to.
I asked it to fetch the recent pull requests from the MLX repository, summarize the changes, and identify anything that needs my attention. The model reasons about the request, calls the GitHub CLI to fetch the PR data, reads through the diffs, and writes a concise summary.
All of this is happening locally. The model runs on my hardware, and only the git commands reach the network. The weights never leave the machine for that summary. The GitHub CLI is the part that touches the network, because the pull requests live there.
That ask is a read and a report. It is the short version of the loop: the model decides, a tool runs, the result comes back, the model writes. The same loop is what writes code later. The path to it is the stack, then the setup, then what the hardware is doing when several agents call at once.
The local agentic AI stack is four layers, from MLX at the bottom up to the agent. Then three steps to point an agent at localhost. Then how MLX uses the hardware so the agents stay fast: prompt processing, several agents at once, and a model that does not fit in one Mac. Then the demos. One is a SwiftUI app from a blank Xcode project. The other is a bug fix from inside Xcode, on the same server.
Four layers, from the bottom
The stack that runs this on the Mac has four layers. I start from the bottom.
At the bottom is MLX, the open-source array framework built for Apple Silicon. It handles the low-level computation, Metal acceleration, and memory management. This is the foundation the rest is built on. The model math happens here, on the GPU, in the unified memory of the machine.
One level up is the language model layer. MLXLM provides what I need to load, run, quantize, and fine-tune a large language model. It supports thousands of models from Hugging Face, and it gives me both CLI tools and a Python API. I can drive a model from the command line, or from Python, without standing up a server yet.
Python is enough to run a model once. To serve an agent I need something more: a persistent server with a standard API. That is MLXLM server.
MLXLM server is an OpenAI-compatible HTTP server. It exposes the local model through a standard API, so the agent does not grow a special client for this Mac. It supports structured tool calling, so the model can invoke a function reliably and take the result back into the loop. It also runs reasoning models that analyze a problem step by step before they respond. It is a drop-in replacement for a cloud LLM API. I bind it to localhost. The port I use is 8080.
At the top of the stack is the agent itself. This can be any framework or tool that speaks the OpenAI chat completions protocol. Xcode, OpenCode, a custom script, or anything else that posts the same request. Because MLXLM server provides a standard interface, any of those works against the local model. The agent posts to the server. The server owns the model.
Ollama, LM Studio, and vLLM build on MLX and MLXLM as well. If you already use one of those, you may already be running on MLX.
Three steps from zero
It takes three steps to go from zero to a fully local agentic workflow.
Step one, install MLXLM. A single pip install gets me what I need to load a model and to serve it.
pip install mlx-lmStep two, start the server. I run MLXLM server with a model that supports tool calling. Starting with a small model, so I can test the setup, is the move I make first. I want to see a full loop before I spend the memory on a large one. The server starts, loads the model, and is ready to accept requests on localhost. I use port 8080.
mlx_lm.server --port 8080The model name is the other argument. It has to be a model the server can run, and it has to be one that can call tools. Without tool calling, the agent can talk, and it cannot run the command or read the file.
Step three, point the agent at the local server. In most agent frameworks I set the base URL to that address, and the wiring is done. The agent does not know or care that the model is running on this Mac. It sends the same chat completion it would send to a cloud API.
OpenCode, pointed at localhost
OpenCode is the concrete case. I define a local provider. I set the URL to localhost, and I set the model name the server expects. I also tell OpenCode to use this local model for the session. That is the whole configuration. Every interaction then runs through the local model.
I point any other agent I want at the same base URL. Xcode is the other one, and the click-path for it is further down, once the drawing app exists. A script that speaks the same chat API is the same idea: post to localhost, port 8080, and read the tool calls that come back.
Prompt processing
With an agent talking to MLX, the first limit I hit is prompt processing.
In this workflow, every time the model receives tool output, it has to process all of that new context before it can reason about the next step. A compiler error, a file, a diff: each one is new tokens, and the model reads them before it picks the next tool. This happens over and over through the agentic loop, and it adds up fast.
Agentic sessions usually run to hundreds of thousands of tokens, and most of those tokens are not generated. They arrived as files and command output. The model wrote the smaller share. The slow part is the read.
That read is a pile of matrix multiplies. The M5 chip has dedicated neural accelerators, and MLX can target them for exactly this work. Neural accelerators make matrix multiplication four times faster on M5 compared to M4. With the multiplication and attention kernels in MLX, that carries over almost exactly to prompt processing. Cutting the prompt processing time means the agent can read a codebase, or process a tool result, almost four times faster.
There is no special argument to set, and there is no code to change. MLX selects the kernel for the hardware in the machine, and it just works. I do not pass a flag to turn the accelerators on.
Several agents, one server
The second limit is concurrency. In practice the agents rarely work alone. This is the part I actually use when I say I run several of them on one Mac.
A common pattern is for an agent to spawn sub-agents, each taking a different part of the problem in parallel. One might be reading documentation, another searching the code, and a third writing tests, all at the same time. That means multiple requests hitting the local model together. One server, port 8080, and more than one conversation in flight.
MLXLM server handles this with continuous batching. It does not take the requests one at a time. It dynamically groups the incoming requests into batches and runs them together on the GPU. A new request can join a batch already in progress, without waiting for the current one to finish.
The sub-agents do not stall waiting in a queue. They get served concurrently, which keeps the whole workflow moving. That is what makes several agents on one Mac practical. They share the model. The server owns the batching. I do not start a second copy of the weights for each agent.
When 512 GB is not enough
The third limit is model size. Sometimes one machine is not enough, even one with 512 GB of RAM, because the model is too large to fit in memory. The most recent DeepSeek model has 1.6 trillion parameters and needs more than 800 GB of memory just for the weights.
MLX can spread that model across Macs connected over Thunderbolt or over Ethernet. For agents this does two things. First, I can run a much larger model that would not fit on a single machine. Second, it parallelizes prompt processing across the machines, which speeds up the agentic loop, since the model can process tool results faster. The same read that dominates a session gets split.
Setting up distributed inference with MLXLM server is straightforward. I launch the server with mlx launch and a host file. The file names the nodes and the type of connection between them, Thunderbolt or Ethernet. The model is started across the machines that are listed. Everything else stays the same: the agent still has one URL, and it still speaks the chat API.
Starting with macOS 26.2, Thunderbolt can use RDMA, which is low latency, high bandwidth communication over that cable. Distributed inference with MLX picks up speed from that link, up to three times with four Macs.
The setup for those machines is documented in Explore Distributed Inference and Training with MLX. I follow that when the weights no longer fit in the Mac I am sitting at.
A SwiftUI drawing app from a blank project
The pull request summary was a simple read and report. The next thing I ask is for the agent to write an entire project from scratch, and then to fix a bug in it.
I start with a blank Xcode project, and I ask the agent to build a small SwiftUI drawing app for the iPad. I do not write the Swift.
The agent first looks at the current directory to see the project structure that is already there. It makes a plan to guide the implementation, and then it gets on to writing the code. Using the agent means I do not copy anything into Xcode, and I do not start the build myself. The agent writes the files, then builds the app, fixing any errors it encounters along the way. The compiler output comes back as a tool result, the model reads it, and the loop goes around again.
The model finishes the first version in a couple of minutes. About two minutes, from the blank project to something I can launch. At the same time I have the project open in Xcode, and I launch the app in the simulator.
What the agent created is a fully working drawing app. Fully functional, from a prompt, on a project that had no drawing code in it. Two minutes is the first version, not the last word. With the agent I can keep iterating until I am happy with the result.
I prefer rounded end caps on the stroke. They look better to me. I ask the agent to add them. The agent edits the code and recompiles the app until it compiles without errors. I run the new version in the simulator. The strokes now have rounded end caps.
All of this happened locally. The model ran through MLXLM server on this Mac, and the agent used standard development tools, xcodebuild among them, to verify and build its work. I did not paste the generated Swift anywhere. The files landed in the project, the build ran, and the simulator showed the stroke.
The same server, inside Xcode
The last demo sits inside the editor I already have open. Same drawing app project, same MLX server, already running. OpenCode does not have to sit in the middle for this one.
I connect Xcode to that server. I open Settings and go to the Intelligence tab. I click Add chat provider and select a locally hosted provider. I set the port to 8080, the port I selected when I launched the server. Any other port would be the same step, as long as it matches the process that is listening. Then Xcode can talk to the local model.
I introduce a bug into the drawing app after it was working, and I ask the model to fix it. Within seconds it identifies the bug and inspects the code around it. Then it writes a fix, and I build and run the app again.
That is a local agent in the Xcode workflow I already use. It reads the project files, it understands the build errors, and it makes a targeted fix. Local means the code never leaves the Mac. The Intelligence tab is pointed at localhost, the same way OpenCode is.
Install, launch, point the agent at it
The stack for running agentic AI locally on this Mac runs from MLX up to the agent. Neural accelerators, continuous batching, and distributed inference are what make it fast enough that I can keep more than one agent on the machine.
To get started I install MLXLM, launch the server, and point the agent at it. A small tool-calling model first, port 8080, then OpenCode or Xcode on that URL. MLX, MLXLM, and the server are open source, and they are available now.
The model stays on the Mac. The time I still pay is the read: every tool result has to go through the model before the next edit. When I split the job, the server batches the agents together, and I am still not paying an API bill.