Production grade end to end open source stack for Voice AI

I built a voice agent from scratch that averages ~400ms end-to-end latency (phone stop → first syllable). That’s with full STT → LLM → TTS in the loop, clean barge-ins, and no precomputed responses. What moved the needle: Voice is a turn-taking problem, not a transcription problem. VAD alone fails; you need semantic end-of-turn detection. The system reduces to one loop: speaking vs listening. The two transitions - cancel instantly on barge-in, respond instantly on end-of-turn - define the experience. STT → LLM → TTS must stream. Sequential pipelines are dead on arrival for natural…

Hey HN, this is Igor, one of the engineers behind Inworld Runtime, which we're releasing today in public preview. We built it to solve the common problem we and our customers had: engineers spend more time on AI ops and plumbing than on actual feature development. This was often due to the challenge of using Python for I/O-bound, high-concurrency workloads and complexity maintaining pipelines with streams that use always-changing ML models. Our solution is a high-performance runtime written in C++ with the core idea of defining AI logic as graphs. For instance, a basic voice-to-voice…

Build Powerful Voice Agents

I've been obsessed for the past ~year with the possibilities of talking to LLMs. I built a bunch of one-off prototypes, shared code on X, started a Meetup group in SF, and co-hosted a big hackathon. It turns out that there are a few low-level problems that everybody building conversational/real-time AI needs to solve on the way to building/shipping something that works well: low-latency media transport, echo cancellation, voice activity detection, phrase endpointing, pipelining data between models/services, handling voice interruptions, swapping out different…

Hi HN! Last year the project I launched here got a lot of good feedback on creating speech to speech AI on the ESP32. Recently I revamped the whole stack, iterated on that feedback and made our project fully open-source—all of the client, hardware, firmware code. This Github repo turns an ESP32-S3 into a realtime AI speech companion using the OpenAI Realtime API, Arduino WebSockets, Deno Edge Functions, and a full-stack web interface. You can talk to your own custom AI character, and it responds instantly. I couldn't find a resource that helped set up a reliable, secure websocket (WSS) AI…

A few years ago, right after high school, I decided to try to make a simultaneous translation app for Android as a side project, it took longer than expected (about 2 years) and I had to make a lot of compromises (I had to use Google's API and therefore make users use a developer key because at the time there were no free solutions for speech recognition and translation that had good quality). At the end of university, I decided to pick it up again and finally, using OpenAi's Whisper for speech recognition and Meta's NLLB for translation (with both running locally on the phone), I managed to…