Top AI Product

Fermion Research took Qwen3-8B, ran ternary quantization-aware training, and distilled out an 8.19B decoder that ships as one 3.88GB file. All 252 linear layers stored in a three-value weight format 8x smaller than fp16, decoded inside the matrix kernels. It hit HackerNews the day it launched, 2026-07-27.

What it actually is

An open-weights language model, not a tool or an app. The numbers hold up for its size — MMLU 72.1, IFEval 77.2, BFCL 68.9 — so this isn’t compression that guts the model to hit a file size. It ships a 0.6B draft model for speculative decoding that only accepts tokens matching the 8B’s argmax, so you get the speedup with output identical to plain greedy.

Running it yourself

Weights are on Hugging Face in a TRTC v4 container. Commercial use, fine-tuning, and redistribution all open under Apache 2.0 — no access request, no form. Since single-stream decode is bound by bytes moved per token, a 3.88GB working set runs faster than a 16GB fp16 build on the same memory, and fits beside its KV cache on an 8GB GPU or a 16GB laptop. That’s the point: a SOTA-class 8B you can drop onto almost any device.

You Might Also Like

• Jamesob Local llm the 2026 Field Manual for Running Sota Models on Your own Hardware
• Mcp2cli the Tool That Cuts mcp Token Costs by 99 Just hit Hacker News
• Hermes Agent by Nous Research Might be the Open Source ai Agent That Finally Remembers Everything
• Cursor Composer 2 Takes on Anthropic and Openai With a 0 50 m Token Coding Model and the Benchmarks Back it up
• 397 Billion Parameters on a 48gb Macbook Flash moe Turns Apples 2023 Research Into Reality