Helicone (YC W23) – OSS LLM Observability and Development Platform
Hey HN, we're Justin and Cole, the founders of Helicone (https://helicone.ai). Helicone is an open-source platform that helps teams build better LLM applications through a complete development lifecycle of logging, evaluation, experimentation, and release. You can try our free demo by signing up (https://helicone.ai/signup) or self-deploy with our new fully open-source helm chart (https://helicone.ai/selfhost). When we first launched 22 months ago, we focused on providing visibility into LLM applications. With just a single line of code, teams could…
In plain words
Helicone is an open-source platform for developing and monitoring large language model applications. It provides logging, evaluation, experimentation, and release management tools accessible through a single line of code integration. Teams can self-host using Helicone's open-source Helm chart or use the hosted demo version, with the platform having processed over 2.1 billion requests for users ranging from startups to enterprise companies.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN, we're Justin and Cole, the founders of Helicone (https://helicone.ai). Helicone is an open-source platform that helps teams build better LLM applications through a complete development lifecycle of logging, evaluation, experimentation, and release. You can try our free demo by signing up (https://helicone.ai/signup) or self-deploy with our new fully open-source helm chart (https://helicone.ai/selfhost). When we first launched 22 months ago, we focused on providing visibility into LLM applications. With just a single line of code, teams could trace requests and responses, track token usage, and debug production issues. That simple integration has since processed over 2.1B requests and 2.6T tokens, working with teams ranging from startups to Fortune 500 companies. However, as we scaled and our customers matured, it became clear that logging alone wasn’t enough to manage production-grade applications. Teams like Cursor and V0 have shown what peak AI application performance looks like and it's our goal to help teams achieve that quality. From speaking with users, we realized our platform was missing the necessary tools to create an iterative improvement loop - prompt management, evaluations, and experimentation. Helicone V1: Log → Review → Release (Hope it works) From talking with our users, we noticed a pattern: while many successfully launch their MVP quickly, the teams that achieve peak performance take a systematic approach to improvement. They identify inconsistent behaviors through evaluation, experiment methodically with prompts, and measure the impact of each change. This observation shaped our new workflow: Helicone V2: Log → Evaluate → Experiment → Review → Release It begins with comprehensive logging, capturing the entire context of an LLM application. Not just prompts and responses, but variables, chain steps, embeddings, tool calls, and vector DB interactions (https://docs.helicone.ai/features/sessions). Yet even with detailed traces, probabilistic systems are notoriously hard to debug at scale. So, we released evaluators (either via LLM-as-judge or custom Python evaluators leveraging the CodeSandbox SDK - https://codesandbox.io/docs/sdk/sandboxes). From there, our users were able to more easily monitor performance and investigate what went wrong. Did the embedding search return poor results? Did a tool call fail? Did the prompt mishandle an edge case? But teams would still edit prompts in a playground, run a few test cases, and deploy based on intuition. This lacked the systematic testing we’re used to in traditional software development. That’s why we built experiments (similar to Anthropic's workbench but model-agnostic) (https://docs.helicone.ai/features/experiments). For instance, when a prompt generates occasional rude support responses, you can test prompt variations against historical conversations. Each variant runs through your production evaluators, measuring real improvement before deployment. Once deployed, the cycle begins again. We recognize that Helicone can’t solve all of the problems you might face when building an LLM application, but we hope that we can help you bring a better product to your customers through our new workflow. If you're curious how our infrastructure handled our growth: Our initial architecture struggled - synchronous log processing overwhelmed our database and query times went from milliseconds to minutes. We've completely rebuilt our infrastructure with two key changes: 1) using Kafka to decouple log ingestion from processing, and 2) splitting storage by access pattern across S3, Kafka, and ClickHouse. This was a long journey but resulted in zero data loss and fast query times even at billions of records. You can read about that here: https://upstash.com/blog/implementing-upstash-kafka-with-clo... We'd love your feedback and questions - join us…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, January 2025
the whole month →- IM
Hello! I'm Byran. I spent the past ~6 months engineering a laptop from scratch. It's fully open-source on GH at: https://github.com/Hello9999901/laptop
Dev tools · 2025 · byran.ee
- TITetris in a PDF▲1,289
I realized that the PDF engines of modern desktop browsers (PDFium and PDF.js) support JavaScript with enough I/O primitives to make a basic game like Tetris. It was a bit tricky to find a union of features that work in both engines, but in the end it turns out that showing/hiding annotation "fields" works well to make monochrome pixels, and keyboard input can be achieved by typing in a text input box. All in all it's quite janky but a nice reminder of how general purpose PDF scripting can be. The linked PDF is all ASCII so you can just open it in a text editor, or have a look at…
Life & fun · 2025 · th0mas.nl
Create lifelike, personalized AI avatars from text prompts
AI · 2025 · jogg.ai
