OpenLLMetry – OpenTelemetry-based observability for LLMs
Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://github.com/traceloop/openllmetry. There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the…
In plain words
OpenLLMetry is an open-source set of extensions built on OpenTelemetry that provides observability for large language model applications. It automatically monitors and traces prompts, token usage, vector databases, embeddings, and other LLM-specific data. Designed for developers building generative AI applications, OpenLLMetry uses an open protocol rather than vendor-specific frameworks, avoiding lock-in to proprietary observability platforms. The project is available on GitHub and remains in early development.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://github.com/traceloop/openllmetry. There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the gen-AI space so we think it’s the right time to define an open protocol for observability. So we built OpenLLMetry. It extends OpenTelemetry and provides instrumentations for LLM-specific libraries which automatically monitor and trace prompts, token usage, embeddings, etc. Two key benefits with OpenTelemetry are (1) you can trace your entire system execution, not just the LLM (so you can see how requests to DBs, or other calls affect the overall result); and (2) you can connect to any monitoring platform—no need to adopt new tools. Install the SDK and plug it into Datadog, Sentry, or both. Or switch between them easily. We’ve already built instrumentations for LLMs like OpenAI, Anthropic and Cohere, vector DBs like Pinecone and LLM Frameworks like LangChain and Haystack. And we’ve built an SDK that makes it easy to use all of these instrumentations in case you’re not too familiar with OpenTelemetry. Everything is written in Python (with Typescript around the corner) and licensed with Apache-2.0. We’re using this SDK for our own platform (Traceloop), but our hope is that OpenLLMetry can evolve and thrive independently, giving everyone (including our users) the power of choice. We’ll be working with the OpenTelemetry community to get this to become a first-class citizen of OpenTelemetry. Would love to hear your thoughts and opinions! Check it out - Docs: https://www.traceloop.com/docs/python-sdk/introduction Github: https://github.com/traceloop/openllmetry
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, October 2023
the whole month →
Nudge 2.0▲1,051In-app experiences to activate, retain, & understand users
Growth · 2023 · nudgenow.com
Unlock AI magic for elevated customer engagement, fast
AI · 2023 · tiledesk.com
- OD
Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website.
Dev tools · 2023 · github.com