A 744B-parameter frontier model answering questions on a machine with 25GB of RAM. No GPU, no Python, no dependencies — one C file, about 2,400 lines. Colibri hit 799 points on Hacker News and 1.8k GitHub stars because it sounds impossible.
How 370GB fits in 25GB
Colibri is a local inference engine for GLM-5.2, a Mixture-of-Experts model that activates only ~40B of its 744B parameters per token. The dense parts — attention, embeddings, shared experts — sit in RAM at int4, about 9.9GB. The other 370GB of expert weights stay on your SSD and stream in on demand through an LRU cache. MLA compression shrinks the KV cache 57×, speculative decoding gets 2.2–2.8 tokens per forward pass, and hand-written AVX2 int8/int4 kernels do the math.
Why it matters
The speed is honest: 0.05–0.1 tokens/s cold, 0.37 warm. Nobody will chat with this. It’s a proof — the real constraint on running frontier models locally is disk bandwidth, not VRAM. Every other runtime assumes the model fits in memory; Colibri only asks that it fit on your SSD. One person built it, on a 12-core laptop.
You Might Also Like
- Opencode Crossed 120k Github Stars and Even Anthropics Legal Threats Couldnt Slow it Down
- Understand Anything Scores 2400 Github Stars by Mapping Codebases With Five ai Agents
- Aio Sandbox Hits 3 8k Github Stars by Giving ai Agents a Proper Computer to Work With
- 708 Github Stars in 48 Hours Claude Token Efficient Universal Claude md and the Fight Over Claudes Most Expensive Habit
- Microsoft mai Code 1 Flash is Live in Github Copilot 60 Fewer Tokens Than Comparable Coding Models
Top AI Product 
Leave a comment