mlx-serve

mlx-serve is a free and open source native application for macOS that enables users to run large language models (LLMs) and generative AI models entirely on Apple Silicon Macs. Designed for privacy and performance, the tool allows users to interact with AI for tasks such as chat, coding agents, image editing, video creation, music composition, 3D model generation, and voice cloning, all processed locally without any data leaving the device.

The platform supports a wide range of AI capabilities, including editing photos with text prompts, animating images into videos with synchronized sound, generating original music, converting photos into textured 3D models, and cloning voices from short audio samples. Coding features such as running Claude Code offline without an API key are also available. Users can summon AI through a launcher, voice commands, or scheduled actions, and sandbox AI agents in a Linux virtual machine to ensure system security. The tool is compatible with OpenAI and Anthropic APIs, allowing existing apps built for these APIs to connect seamlessly to mlx-serve. It also natively supports Ollama's protocol, enabling integration with tools designed for Ollama without modification.

Performance is a key focus, with benchmarks indicating that mlx-serve operates faster than comparable tools like LM Studio on identical hardware and models. The application leverages speculative decoding and prompt lookup decoding to increase token generation speed, with adaptive mechanisms to maintain output quality. It supports running advanced models such as DeepSeek V4 Flash with up to 284 billion parameters on Macs equipped with sufficient unified memory. The software is distributed as a single self-contained binary, requiring no external dependencies or setup scripts, and includes features such as eager warmup for quick initial responses and fast conversational turnarounds.

mlx-serve is available for Apple Silicon Macs running macOS 26 or later (M1 through M4). It is open source, downloadable at no cost, and does not require accounts, subscriptions, or cloud calls, making it suitable for users seeking private, high-performance local AI capabilities.