Top AI Product

Local coding agents have one dealbreaker on a Mac: every turn, the model recomputes the entire conversation history before writing a single token. Ninety seconds of waiting, per turn. oMLX exists to kill exactly that.

It’s a macOS menu bar app — native Swift, not Electron — that turns Apple Silicon into a local LLM inference server. Apache 2.0, 21k GitHub stars, built on MLX, serving text, vision, OCR, embedding and reranker models with continuous batching.

The cache everyone else throws away

Most local tools discard the KV cache after each request, which is why local agents feel unusable. oMLX tiers it instead: hot blocks in RAM, cold blocks on SSD — and the cache survives restarts. Multi-turn agent sessions drop from 90-second waits to about 5. It’s the rare local project that gets this one thing right.

Two APIs, zero migration

The server speaks both OpenAI and Anthropic API formats with streaming and tool calling, so Claude Code, Cursor, or any existing client just points at localhost. Typical setup: keep a local Qwen warm for your coding agent, run embeddings and reranking for local RAG — one server, sitting quietly in your menu bar.

You Might Also Like

• Alibabas Agentscope Hits 21k Github Stars What Makes This Multi Agent Framework Different
• Openai Codex Record and Replay Demo a mac Task Once the Agent Repeats it Forever
• Meta Model api Muse Spark 1 1 Undercuts Openai and Anthropic at 1 25 4 25 per Million Tokens
• Google Gemini Enterprise Gives Every ai Agent a Cryptographic id its Real Fight With Openai and Anthropic
• Openfang Just Dropped and its Already the Hottest Agent os on Github