Alternatives
Products that do what TraceOps does
TraceOps! Understand every decision your LLM makes.
- 1
- 2
- 3
- 4OO
Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://github.com/traceloop/openllmetry. There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the…
2023 · github.com
- 5
- 6
- 7
- 8OA
Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…
Jul 2026 · oodle.ai
- 9
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 10YD
If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
2024 · github.com
- 11
- 12
- 132C
Single-agent LLMs suck at long-running complex tasks. We’ve open-sourced a multi-agent orchestrator that we’ve been using to handle long-running LLM tasks. We found that single LLM agents tend to stall, loop, or generate non-compiling code, so we built a harness for agents to coordinate over shared context while work is in progress. How it works: 1. Orchestrator agent that manages task decomposition 2. Sub-agents for parallel work 3. Subscriptions to task state and progress 4. Real-time sharing of intermediate discoveries between agents We tested this on a Putnam-level math problem, but the…
Feb 2026 · github.com
- 14RB
We built HALO (Hierarchal Agent Loop Optimizer), an open-source tool for debugging and optimizing AI agents using their execution traces. It’s a loop. Run your agent, feed the traces to HALO, get the report, apply the fixes, then re-run your agent. HALO takes in OTEL compliant traces from AI agents using tracing frameworks such as Langfuse, Arize/OpenInference, or even just plain JSONL. It uses an RLM (Recursive Language Model) to more efficiently break trace analysis into smaller subproblems in order to find recurring patterns across large amounts of data and fix systemic issues that…
Jun 2026 · github.com
- 15MY
LLM observability is an absolute must-have for anyone running something in prod (or prod-like). While all the observability startups are great, you're essentially sending all your OpenAI usage history - prompts, generations, chats - to a random third party. So this script deploys a basic proxy in your Azure account, catches all incoming OpenAI requests, stores logs in your own resource group, and comes with visualizations premade (charts, timelines, chat history, cost estimation, etc). Thanks for any thoughts and feedback!
2023 · github.com
- 16OS
Hi HN, Hugh and Vince here. LLMonitor helps you record, trace & search your LLM queries and chatbot conversations. You can also capture user feedback on your frontend and correlate it with backend LLM queries then use that to fine-tune your own models. The project started has an internal tool in our previous (failed) AI startup. We’re aware the LLM observability space is very crowded. Apart from being open-source, we differentiate with: - Model-agnostic and minimal lock-in (no MITM of requests). - High focus on DX and dashboard clarity. - Support for complex scenarios: e.g. a chatbot that…
2023 · github.com
- 17TO
Hey Xinwei and Zecheng here, we are the authors of TraceRoot (https://github.com/traceroot-ai/traceroot). TraceRoot (https://traceroot.ai) is an open-source debugging platform that helps engineers fix production issues faster by combining structured traces, logs, source code contexts and discussions in Github PRs, issues and Slack channels, etc. with AI Agents. At the heart are our lightweight Python (https://github.com/traceroot-ai/traceroot-sdk) and TypeScript (https://github.com/traceroot-ai/traceroot-sdk-ts) SDKs -…
2025 · github.com
- 18TA
Hi HN, There’s been a lot of discussion lately around context graphs, decision traces, and how AI systems reason. One thing we kept running into: when AI agents make real decisions, the why behind those decisions often disappears. The context is scattered across prompts, tools, policies, and approvals. Logs show what happened, but not why it was allowed. TraceMem is an attempt to make decision context durable. It records the reasoning, authority, and context behind AI actions as a system of record, not as monitoring data, but as memory. Happy to share more details or answer questions. - Tommi
Jan 2026 · tracemem.com
- 19LA
We combined Stanford's ACE (agents learning from execution feedback) with the Reflective Language Model pattern. Instead of reading traces in a single pass, an LLM writes and runs Python in a sandbox to programmatically explore them - finding cross-trace patterns that single-pass analysis misses. The framework achieved 2x consistency improvement on τ2-bench.
Mar 2026 · github.com
- 20PO
We are the developers of Phoenix, which we released in April of this year with a goal of bringing LLM observability to the notebook. In the time since, the growth of LLM frameworks and complex agent workflows led us to add support for LLM spans and traces and introduce a simple Eval harness for testing the data from those spans. The latest Traces & Spans release of Phoenix offers: -Out of the box tracing for LlamaIndex and LangChain -Fully local execution, no data sent anywhere, outside of your own LLM calls -Ability to get a common dataframe format across frameworks back to a notebook for…
2023 · github.com
- 21IB
Excited to share a project I’ve been building for months! Would love to receive honest feedback :) My motivation: AI is clearly going to be the interface for data. But earlier attempts (text-to-SQL, etc.) fell short — they treated it like magic. The space has matured: teams now realize that AI + data needs structure, context, and rules. So I built a product to help teams deliver “chat with data” solutions fast with full control and observability (agent tracing, quality scores, etc) — am I wrong? The product allows you to connect any LLM to any data source with centralized context…
Oct 2025 · github.com
- 22TO
I built TraceAIO, an open-source tool that prompts LLMs on your behalf and tells you whether ChatGPT, Perplexity, and Gemini mention your brand — and which competitors and sources show up instead. Yeah, this category smells a bit like a grift, same as early SEO. And I think over time it will become just SEO again, and become about good content. The tool just helps you monitor over time. It queries the browser products through real browser sessions, not APIs, runs on Docker, with an MCP server so you can query your own data through an LLM. No business model, Apache 2.0, self hosted. If you…
Jun 2026 · traceaio.org
- 23
Build autonomous Python agents with native Agent-to-Agent (A2A) communication - protolink/examples/ai_courtroom at main · nMaroulis/protolink
28d ago · github.com
- 24WI
At Laminar (https://github.com/lmnr-ai/lmnr) we're building open source AI observability platform in Rust. We obsess over instrumentation DX for our Python and TS SDKs and in this new blog we outline how we made the most seamless way of instrumenting recently released claude agent sdk
Dec 2025 · laminar.sh
Ranked by how close each launch is in meaning, then by votes. Refine with a description →