Local Mac Server Lets Developers Run Open Models Without Python or Cloud 🔗
One Zig-built server speaks OpenAI, Anthropic and Ollama APIs, adding image, video, music and speech generation on Apple Silicon.
mlx-serve lets developers run open-weight models entirely on an Apple Silicon Mac, with no Python runtime, no account and no network dependency. The server listens on localhost:11234 and answers requests written for the OpenAI, Anthropic and Ollama APIs, so existing clients can be pointed at it without code changes.
The backend is written in Zig, with a Swift macOS app layered on top for chat and generation. The project's own release notes highlight the scope beyond text:
- Coding agents:
mlx-serve launch claudestarts Claude Code against a local model, configured to the model's real context window. - Media generation: image, video, music and speech models, including FLUX.2 and Qwen3-TTS with voice cloning, run in the app or over HTTP.
- Large models: the README says some mixture-of-experts models larger than system memory can run on a 128 GB Mac by streaming experts from SSD, and two Macs linked by Thunderbolt 5 can share one model.
For builders, the appeal is a single local endpoint that replaces a cloud API for development, private transcription and offline assistants, under an MIT license. The project also ships Linux builds for NVIDIA GPUs and an iPhone build.
The catch: The project is roughly eight months old and carries 67 open issues, and its quality on media-generation models is not independently benchmarked here. It requires macOS 26.2 on Apple Silicon, so it is no help to Intel or Windows developers, and anyone adopting it for production should verify how the SSD-streaming path performs on their own hardware before relying on it.
Why this leads today It offers developers a new way to run local models on their own Macs, plugging into existing OpenAI and Anthropic tooling without Python, while today's other contenders are niche or long established.
- Mac developers running Claude Code or Codex against a local model offline.
- Teams keeping transcription and meeting notes on-device for privacy reasons.
- App builders swapping a cloud LLM endpoint for a local OpenAI-compatible server.
Source: ddalcu/mlx-serve — based on the README and release notes.