Open-source observability for LLM apps
Hi HN, Hugh and Vince here. LLMonitor helps you record, trace & search your LLM queries and chatbot conversations. You can also capture user feedback on your frontend and correlate it with backend LLM queries then use that to fine-tune your own models. The project started has an internal tool in our previous (failed) AI startup. We’re aware the LLM observability space is very crowded. Apart from being open-source, we differentiate with: - Model-agnostic and minimal lock-in (no MITM of requests). - High focus on DX and dashboard clarity. - Support for complex scenarios: e.g. a chatbot that…
Open-source observability for LLM apps launched on October 28, 2023 with 10 votes, #1,476 of 4,393 launches that month and more than 75% of that year's launches. The site was not reachable when checked in September 2026.
In plain words
LLMonitor is an open-source observability platform that records, traces, and searches LLM queries and chatbot conversations. Users can capture customer feedback and correlate it with backend queries to fine-tune models. The tool works model-agnostic without intercepting requests, emphasizes dashboard clarity, and handles complex scenarios with multiple agents and tools. It's designed for early-stage startups and indie developers building LLM applications.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN, Hugh and Vince here. LLMonitor helps you record, trace & search your LLM queries and chatbot conversations. You can also capture user feedback on your frontend and correlate it with backend LLM queries then use that to fine-tune your own models. The project started has an internal tool in our previous (failed)…
Does a similar job
all alternatives →
AI Observability by OpenObserveAug 2026 · openobserve.ai · ▲317OpenTelemetry-native observability for agents and LLMs
OpenLLMetry – OpenTelemetry-based observability for LLMs2023 · github.com · ▲154Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://github.com/traceloop/openllmetry. There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the…
You don't need to adopt new tools for LLM observability2024 · github.com · ▲102If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, October 2023
the whole month →
Nudge 2.0▲1,051In-app experiences to activate, retain, & understand users
Growth · 2023 · nudgenow.com
Unlock AI magic for elevated customer engagement, fast
AI · 2023 · tiledesk.com
Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website.
Dev tools · 2023 · github.com