OpenAI’s Realtime API gives you a voice agent, but you rent it and your audio leaves the building. Hugging Face just shipped the open-source answer: speech-to-speech, a framework that chains VAD (Silero v5) → STT → LLM → TTS into one parallel pipeline and exposes a WebSocket API that’s drop-in compatible with OpenAI Realtime. Point your existing client at localhost and it works. 8.2k GitHub stars, +827 today — the fastest-climbing voice repo right now.
Every part swaps out
Nothing is locked. STT runs Parakeet TDT, Whisper, or Paraformer. TTS runs Qwen3-TTS, Kokoro, or Pocket TTS. The LLM plugs into vLLM, llama.cpp, or OpenRouter if you want a hosted brain. Mic, TCP, or custom WebSocket for input. Mix and match until latency and voice quality land where you need them.
Why it matters
This isn’t a demo. It already runs the conversation backend for thousands of Reachy Mini robots in production. For anyone building voice agents who can’t send customer audio to a third party — healthcare, hardware, on-device — this is a real path: the Realtime API shape, fully local, zero per-minute bill.
You Might Also Like
- Openai gpt Realtime 2 Translate Whisper Three Voice Models one api Several Startups Erased
- Openai gpt Realtime 2 1 gpt Realtime 2 1 Mini cut Voice Agent Latency by 25
- Ggml Llama cpp Joins Hugging Face and Honestly it was Only a Matter of Time
- Tradingagents 33k Stars for an Open Source Framework That Simulates an Entire Trading Firm With llm Agents
- Pollen Robotics Reachy Mini a 299 Desktop Humanoid That Runs 1 7m Hugging Face Models
Top AI Product 
Leave a comment