Live conversations with ChatGPT using WebRTC
HN, meet KITT! https://livekit.io/kitt Like many folks here, the LiveKit team is enamored with ChatGPT. Given that we spend most of our time working with real-time media, we thought we'd try connecting GPT to a WebRTC video call. KITT can do some neat things: - Answer questions like Siri, Alexa, or Google Assistant - Summarize what was discussed in a meeting - Speak multiple languages and even act like a third-party translator - Act as a DM in a D&D campaign At first, we weren’t sure if we could get the latency low enough to have a human-like conversation, but after making a…
What it does
In the maker’s words, at launch
HN, meet KITT! https://livekit.io/kitt Like many folks here, the LiveKit team is enamored with ChatGPT. Given that we spend most of our time working with real-time media, we thought we'd try connecting GPT to a WebRTC video call. KITT can do some neat things: - Answer questions like Siri, Alexa, or Google Assistant - Summarize what was discussed in a meeting - Speak multiple languages and even act like a third-party translator - Act as a DM in a D&D campaign At first, we weren’t sure if we could get the latency low enough to have a human-like conversation, but after making a handful of tweaks, things feel pretty close to speaking with a person. The key optimization we made was to stream all the things: - We convert streaming audio from participants to text in 20ms frames - We pre-prompt GPT to be concise in its responses and generate short sentences - Each sentence is converted to speech in real-time and streamed out to all participants We also use GPT-3 Turbo instead of GPT-4 which shaves off response time, as well. To make it easy for anyone to plug in their own AI, we built KITT as a server-side Go program that uses [Pion](https://github.com/pion/webrtc) to publish audio and video streams like any other WebRTC participant. That means it’s fairly straightforward to plug in your own STT, LLM, custom voice or avatar. For more details on how we built this: https://blog.livekit.io/meet-kitt Would love to hear your thoughts and feedback in the comments!
Does the same job
all alternatives →
Omni Channel Custom GPT Chatbot 2023 · ▲124Create GPT chatbots for your data & publish on all platforms
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, April 2023
the whole month →
- G4
Hi HN, Today we’re launching GPT-4 answers on Phind.com, a developer-focused search engine that uses generative AI to browse the web and answer technical questions, complete with code examples and detailed explanations. Unlike vanilla GPT-4, Phind feeds in relevant websites and technical documentation, reducing the model’s hallucination and keeping it up-to-date. To use it, simply enable the “Expert” toggle before doing a search. GPT-4 is making a night-and-day difference in terms of answer quality. For a question like “How can I RLHF a LLaMa model”, Phind in Expert mode delivers a…
AI · 2023 · phind.com
