Top AI Product

QLoRA cut fine-tuning VRAM roughly in half. Soup CLI removes the assumption underneath: your GPU never holds the whole model.

It’s an open-source fine-tuning CLI — one YAML, one command, full LoRA run. The core is layer streaming: the frozen base model lives in CPU RAM, one decoder layer at a time streams into a small VRAM buffer, and only the LoRA adapters stay resident on GPU. Stack NF4 4-bit quantization on top, and peak VRAM is set by one layer, not the model. On a 4GB RTX 3050 laptop, Llama-3.1-8B trains at 119.6 tok/s in 3.32GB — bit-exact against a normal resident run.

Built to be scripted

pip install "soup-cli[train]", write a YAML (base model, task, quantization), run. Nine architectures — Llama, Qwen, Mistral, Gemma, Phi — plus a VRAM pre-check before training and NVMe offload when RAM runs out. YAML-driven means it drops straight into scripts and CI: automated fine-tunes on the cheapest hardware you own.

Why it hit both leaderboards

Show HN and Product Hunt front pages the same week, 414 GitHub stars and climbing. The pitch is verifiable: solo dev Alpamys Makazhan published exact numbers anyone with a gaming laptop can reproduce. Unsloth makes fine-tuning faster on GPUs that qualify; Soup makes it work on ones that don’t.

You Might Also Like

• Hypura Runs a 31gb Model on a 32gb mac at 2 2 tok s Llama cpp Just Ooms
• Unsloth Studio Brings no Code llm Fine Tuning to Your Laptop 2x Faster 70 Less Vram
• Prismml Exits Stealth With 16m and a 1 bit Model That Rivals Llama 3 at 1 16th the Memory
• Ramp x Prime Intellect a 500 rl Fine Tune of a 9b Open Model Beat Every Frontier Config
• Starnus Just hit 1 on Product Hunt and Yeah its Worth the Hype