Run Local LLMs on a Mac With MLX Instead of the Default llama.cpp Engine

If you've installed Ollama on a Mac, pulled a model and figured you're now running AI locally, you are. Chances are you're doing it on the wrong engine, though, and giving up anywhere from 15% to 3x of the machine's speed without ever seeing it happen.

The default path

Almost everybody follows the same route. Install Ollama, run ollama run llama3.2 (or whatever model is trending that week), watch the tokens stream, and assume that's the ceiling on a Mac.

For a long time that was a fair assumption. Under the hood Ollama wraps llama.cpp, a C/C++ inference engine that runs on practically anything: Windows, Linux, Intel Macs, even a Raspberry Pi. It's excellent engineering. It just wasn't designed around how Apple Silicon works.

Apple has had its own machine learning framework, MLX, since late 2023, and it's built around the one thing that sets M-series chips apart: unified memory. The CPU, GPU and Neural Engine all draw on a single shared pool. The Metal backend in llama.cpp has to work around that. MLX was written for it from day one. The distance between "runs on Apple Silicon" and "made for Apple Silicon" turns out to be larger than most people guessed.

When Ollama itself agreed

MLX was easy to wave off as an enthusiast project, something people mentioned on local-LLM forums. That changed in March 2026, when Ollama shipped an MLX backend option for Apple Silicon. In effect the most popular local-LLM tool admitted that its default engine isn't the fastest choice on the hardware most of its Mac users own. Reason enough to look at actual numbers instead of trusting anyone's word.

What switching buys you

The gap varies a lot with model size, architecture and your exact chip, but independent tests point the same way:

• Mid-size mixture-of-experts models, such as the Qwen3-Coder-30B class, on an M4 Pro Mac mini: about 130 tokens per second with MLX against 43 through Ollama's llama.cpp backend. That's 3x, far past rounding error.
• Dense models under roughly 14B parameters: MLX runs 20% to 87% faster than the llama.cpp Metal path.
• Above roughly 27B parameters: the difference shrinks and mostly vanishes. At that size you're limited by memory bandwidth, not compute, so RAM speed matters more than the engine.
• Time to first token, the irritating pause before streaming begins, also improves under MLX, particularly on M5-generation chips with dedicated Neural Accelerators.

So the fair summary isn't that MLX always wins. It wins by a wide margin on the model sizes most people really run locally, and the advantage fades as models get huge.

Three ways to try it

Stay in Ollama and change the backend

With Ollama 0.19 or newer you can opt into the MLX backend without leaving your current setup. It takes the least effort, and it lets you compare on your own hardware before you commit to anything.

Go native with mlx-lm

Install the package and generate straight from the command line:

pip install mlx-lm
python -m mlx_lm.generate --model mlx-community/Qwen3.5-35B-A3B-MLX-4bit --prompt "your prompt here"

Models under the mlx-community organization on Hugging Face come pre-quantized for this. It's the route for maximum speed, as long as you don't mind living a bit nearer the terminal.

Use LM Studio

If Python is a non-starter, LM Studio now supports MLX models through a one-click toggle in its model browser. That's the easiest on-ramp when the mlx-lm option sounds like too much setup.

Pitfalls when switching

• It isn't a plain model swap. MLX needs its own quantized versions of models (the mlx-community repos), not the GGUF files you've been feeding Ollama. Point mlx-lm at a GGUF and it simply fails.
• RAM headroom still counts. Unified memory is efficient, but on a machine already near its limit, say a 16GB M-series Mac, you have to pick the quantization level with care. For machines with less than 32GB of unified memory, 4-bit tends to be the best trade.
• A 70B model shows no gain. That matches the convergence above roughly 27B: when you load the biggest model your Mac can technically hold, the bandwidth ceiling stays the same whichever engine runs it.
• Keep Ollama around. For quick one-off tests, tool-calling workflows and vision models, its ecosystem and library are still more convenient. The sensible move is to stop defaulting to it for anything performance-sensitive, not to uninstall it.

Who it's worth it for

• Intel Macs: skip it. MLX needs Apple Silicon, M1 or later, so stay on llama.cpp and Ollama.
• Base M1 or M2 with 8 to 16GB of RAM: expect a real but smaller improvement. Gains grow with GPU core count and memory bandwidth, so a base M1 won't feel like an M4 Max.
• Pro, Max and Ultra chips running models from about 3B to 20B: the switch pays off most obviously and immediately here.
• Anyone running 30B-plus MoE models: the biggest win category in the benchmarks above.
• Anyone above about 27B dense parameters: switch for the small time-to-first-token improvement, but don't count on the headline figures.

Match the engine to the chip

None of this makes MLX better in some abstract sense. It's about using a tool on the hardware it was designed for. llama.cpp is brilliant, portable software, and that portability is exactly why it can't be tuned as tightly to one chip family as a framework from the company that builds the chip.

If you take local models seriously on a Mac, for privacy, cost or control, the engine under your prompt matters as much as the model you picked. People spend hours choosing which model to download and never ask what's actually running it. Check yours, because there may be more performance sitting there than you think.