How to run a local LLM on a Mac

What do you need to run a local LLM on a Mac?

Two things: Apple Silicon (M1 or later — unified memory lets the GPU address the whole RAM pool, which is why Macs punch above their weight here) and enough RAM for the model you want. Rules of thumb for 4-bit quantized models:

Rules of thumb, not laws — quantization, context length, and what else is running all move the line.

A model actively generating will use GPU and battery; that's the physics of local AI on any tool below.

Option 1: Ollama — the developer default

Ollama (open source, MIT) is the most common starting point. Install it, then pull and run a model from its large library — Llama, Gemma, Qwen, Mistral, DeepSeek, and more:

# install, then:
$ ollama run gemma3
# it downloads the model, then drops you into a chat

You get a CLI, a local REST API on port 11434 that a huge ecosystem of chat UIs and libraries builds against, and a desktop chat app. Best for: developers, scripting, and anyone who wants maximum model choice. What it doesn't do: voice, document memory/RAG, or agent tools — those you assemble from the ecosystem.

Option 2: LM Studio — the GUI model browser

LM Studio is a free desktop app with a polished GUI: browse and download models, chat with them, chat with your documents, and serve an OpenAI-compatible local API. On Apple Silicon it can run MLX-format models as well as GGUF, which makes good use of the hardware. The app itself is proprietary (its CLI and SDK are open source), and it's cross-platform. Best for: people who want to explore many models without touching a terminal. What it doesn't do: voice, deep OS integration, or an always-on assistant experience.

Option 3: MLX and mlx-lm — Apple's own framework

MLX is Apple's open-source array framework for Apple Silicon; mlx-lm is its LLM toolkit. This is the layer real Mac-native AI apps are built on, and you can drive it directly from Python:

$ pip install mlx-lm
$ mlx_lm.generate --model mlx-community/Qwen3-4B-4bit \
    --prompt "Explain unified memory in one paragraph."

Thousands of pre-quantized models live under mlx-community on Hugging Face. Best for: Python developers, researchers, fine-tuners, and the curious. What it doesn't do: anything product-shaped — it's a framework, and that's the point.

Option 4: M1K3 — a finished local AI companion

The three options above give you a model. M1K3 (free, open source, Apache-2.0, by Round Tower) gives you the assistant: a native SwiftUI app for Apple Silicon Macs that runs Gemma and Qwen via MLX-Swift — the same MLX from option 3, in Swift, on the Metal GPU — plus Apple Foundation Models for instant turns, and builds the rest of the experience around them:

• Live voice — streaming on-device speech recognition and natural speech out.
• A knowledge graph with RAG — drop in documents; get answers grounded in your own files.
• Consent-gated call memory — encrypted, on-device transcription.
• A local agent and an MCP server — so Claude and other agents can query your local knowledge.

First run starts on an instant brain with nothing to download; larger local brains are an opt-in download. Zero bytes of your data go to any server — the optional web-search tool is off by default. The honest catch: it requires macOS 26 Tahoe on Apple Silicon, and its model list is curated rather than a library. Best for: people who want private, working AI on their Mac today, not a stack to assemble.

Which route should you pick?

These aren't rivals so much as layers — M1K3 is literally built on MLX, and plenty of people run Ollama for work and a companion app for everything else. See the full M1K3 vs Ollama comparison.

Do local models match ChatGPT?

No — and this guide is more useful if we say that plainly. Frontier cloud models are ahead of anything a laptop runs, particularly for hard reasoning and state-of-the-art coding. What local models offer instead: they work offline, cost nothing per query, and your data never leaves the machine — privacy by architecture rather than by policy. For summarising, drafting, voice interaction, and Q&A over your own documents, the current Gemma/Qwen class is genuinely good. Full argument: private ChatGPT alternative for Mac.

Frequently asked questions