Alternatives
Products that do what LLMonitor does
Open source monitoring and production toolkit for AI apps
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 10
Trace, evaluate, and improve AI agents in production
30d ago · telerik.com
- 11
- 12
- 13
- 14OA
2016 · oswatch.net
- 15
- 16
- 17
- 18
- 19MY
LLM observability is an absolute must-have for anyone running something in prod (or prod-like). While all the observability startups are great, you're essentially sending all your OpenAI usage history - prompts, generations, chats - to a random third party. So this script deploys a basic proxy in your Azure account, catches all incoming OpenAI requests, stores logs in your own resource group, and comes with visualizations premade (charts, timelines, chat history, cost estimation, etc). Thanks for any thoughts and feedback!
2023 · github.com
- 20
- 21
- 22
- 23
- 24LR
Hi hacker news, My name is Dillion and I'm the creator of llm.report. A few months ago, I was frustrated by the lack of observability into the OpenAI API. All of us are left in the dark about API performance, latency, cost calculation, cost breakdown, and more. I just wanted to know more about how my AI app is performing in production and make data-driven decisions to improve the product. So I ended up just building it myself. There are three parts to the platform: 1. OpenAI API Dashboard (no-code) - Enter your OpenAI key and get access to detailed insights straight from the OpenAI API…
2023 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →