Voice bots with 500ms response times
Last year when GPT-4 was released I started making lots of little voice + LLM experiments. Voice interfaces are fun; there are several interesting new problem spaces to explore. I'm convinced that voice is going to be a bigger and bigger part of how we all interact with generative AI. But one thing that's hard, today, is building voice bots that respond as quickly as humans do in conversation. A 500ms voice-to-voice response time is just barely possible with today's AI models. You can get down to 500ms if you: host transcription, LLM inference, and voice generation all together in one place;…
Voice bots with 500ms response times launched on June 26, 2024 with 315 votes, #127 of 4,672 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Voice bots with 500ms response times is a system for building conversational voice AI agents that respond nearly as quickly as humans. It combines speech transcription, language model inference, and voice generation on a single server to minimize latency. The creator provides a demo and deployable container for running the system on high-end GPUs, addressing the technical challenge of achieving fast, natural-sounding voice interactions with generative AI models.
written from the facts on this page · September 2026
From the sources
A demo showcasing the potential capabilities of voice-driven AI chatbots when optimized and deployed to minimize network and model latency.
In the maker’s words, at launch
Last year when GPT-4 was released I started making lots of little voice + LLM experiments. Voice interfaces are fun; there are several interesting new problem spaces to explore. I'm convinced that voice is going to be a bigger and bigger part of how we all interact with generative AI. But one thing that's hard…
Does a similar job
all alternatives →
Real-time AI Voice Chat at ~500ms Latency2025 · github.com · ▲524Have a natural, spoken conversation with AI! Contribute to KoljaB/RealtimeVoiceChat development by creating an account on GitHub.
gpt-realtime-1.5 by OpenAIFeb 2026 · developers.openai.com · ▲300Tighter instruction adherence in speech agents
I built a sub-500ms latency voice agent from scratchMar 2026 · ntik.me · ▲570I built a voice agent from scratch that averages ~400ms end-to-end latency (phone stop → first syllable). That’s with full STT → LLM → TTS in the loop, clean barge-ins, and no precomputed responses. What moved the needle: Voice is a turn-taking problem, not a transcription problem. VAD alone fails; you need semantic end-of-turn detection. The system reduces to one loop: speaking vs listening. The two transitions - cancel instantly on barge-in, respond instantly on end-of-turn - define the experience. STT → LLM → TTS must stream. Sequential pipelines are dead on arrival for natural…
Conversational AI 2.0 From ElevenLabs2025 · elevenlabs.io · ▲652Luke from ElevenLabs here. We just released Scribe Realtime v2, our new real-time Speech to Text model. We rebuilt it from the ground up around a new architecture that uses predictive transcription – it anticipates the next likely word before it’s spoken. That makes it both fast (around 150 ms latency) and highly accurate. Scribe v2 Realtime outperforms every low latency transcription model across 30 commonly used EU and Asian languages – 93.5% accuracy, compared to Gemini 2.5 Flash (91.4%), GPT-4o MiniTranscribe (90.7%), and Deepgram Nova 3 (85%). It supports 90+ languages including…
I built a tiny LLM to demystify how language models workApr 2026 · github.com · ▲915Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
A personalised AI tutor with < 1s voice responses2024 · educationbot.cerebrium.ai · ▲72TLDR: We created a personalised Andrej Karpathy tutor that can response to questions about his Youtube videos in sub 1 second responses (voice-to-voice). We do this using a voice enabled RAG agent. See later in the post for demo link, Github Repo and blog write up. A few weeks ago we released the worlds fastest voice bot, achieving 500ms voice-to-voice response times, including a 200ms delay waiting for a user to stop speaking. After reaching the front page of HN, we thought about how we could take this a step further based on feedback we were getting from the community. Many companies were…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, June 2024
the whole month →
La Growth Machine▲1,216Create personalized, multi-channel conversations at scale
AI · 2024 · lagrowthmachine.com
PyjamaHR▲1,132Hiring on autopilot. The AI applicant tracking system (ATS).
Work · 2024 · pyjamahr.com