Top AI Product

n0, the team behind P2P networking library iroh, shipped Mesh LLM on July 11. It grabbed 147 points on the Hacker News front page in a day. The pitch: your gaming PC, your MacBook, that old workstation — stitched into one inference cluster, no cloud involved.

How the mesh actually works

This is local-first inference infra, not another API wrapper. Machines find each other through gossip discovery over iroh’s QUIC network, NAT hole-punching included — even behind different routers. A request runs on your local GPU, routes to a peer that already has the model loaded, or triggers “Skippy” mode: the model gets sliced layer by layer across machines, so several modest computers collectively run something none could hold alone. 40+ models supported, from 0.5B up to 235B MoE giants — in an 18MB binary.

One endpoint, zero code changes

Everything surfaces as an OpenAI-compatible API at localhost:9337/v1. Point any existing client or agent framework at it and your stack goes fully private — the obvious use case is agents and RAG over data you’d never send to a cloud API. ACP agent support and a Swift SDK for mobile are on the roadmap. iroh already runs on hundreds of thousands of devices in production — this is real plumbing, not a weekend demo.

You Might Also Like

• Openai Oauth Turns Your Chatgpt Subscription Into a Free Openai api but Should you use it
• Openai gpt Realtime 2 Translate Whisper Three Voice Models one api Several Startups Erased
• Meta Model api Muse Spark 1 1 Undercuts Openai and Anthropic at 1 25 4 25 per Million Tokens
• Gpt oss 120b Openai Finally Goes Open Source and its Worth the Wait
• Llm Skirmish What Happens When you let ai Models Fight Each Other in an rts Game