AI · alternatives · 2026
24 alternatives to Auditi
I've been building AI agents at work and the hardest part isn't the prompts or orchestration – it's answering "is this agent actually good?" in production. Tracing tells you what happened. But I wanted to know how well it happened. So I built Auditi – it captures your LLM traces and spans and automatically evaluates them with LLM-as-a-judge + human annotation workflows. Two lines to get started: auditi.init(api_key="...") auditi.instrument() # monkey-patches OpenAI/Anthropic/Gemini Every API call is captured with full span trees, token usage, and costs. No code changes to your…
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
traceAI▲273Open-source LLM tracing that speaks GenAI, not HTTP.
Apr 2026 · github.com · its alternatives →
- 2
OpenTelemetry-native observability for agents and LLMs
Aug 2026 · openobserve.ai · its alternatives →
- 3
Hi HN! We’re the team behind Fiddler, and we recently built an Agent Cache feature. It works by capturing an LLM request/response pair and, once caching is enabled, serving subsequent matching calls locally so duplicate requests never hit the provider. It’s meant to reduce three pains that show up together in agent development: token cost, feedback-loop latency, and output non-determinism. We’d love to hear your thoughts.
Aug 2026 · telerik.com · its alternatives →
- 4
Trails▲81Hey all, Justin here. I previously built Phind, the AI search engine for developers. One of the biggest problems we had there was figuring out what went wrong with bad searches. We had tons of searches per day, but less than 1% of users gave any explicit feedback. So we were either manually digging through searches or making general system improvements and hoping they helped. This problem gets harder with agents. Traces are longer and more complex. It takes more effort to review them, so I'm building a tool that lets you analyze LLM outputs directly to help developers of LLM apps and agents…
Jan 2026 · trails-red.vercel.app · its alternatives →
- 5
Retracefree tier, from $29/mo▲101Retrace records your AI agents runs so you can replay them step by step, fork from any point to a fix, and share the result as a link
Jul 2026 · retraceai.tech · its alternatives →
- 6
- 7Langtrace AI▲119
Improve your LLM apps with open-source observability tool
2024 · langtrace.ai · its alternatives →
- 8
- 9
I started leaning in on AI heavily this year, as I wanted to get more done autonomously, but then my token usage climbed dramatically to the point where my weekly quota would run out before the end of the week, sometimes a couple of days into the week. I realised I had to do something about it else I'd have to double my spend. So I decided to start tracking my cost per task type. This revealed that a lot of my spend went to searches/scans or simple things like scouting tasks. I then decided to turn this into a simple CLI tool that can be used to read your OpenAI-style logs locally, and…
Jul 2026 · github.com · its alternatives →
- 10AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
- 11
Tracea▲80Datadog for AI agents with traces, RCA, and team memory
May 2026 · tracea.dev · its alternatives →
- 12
RagMetrics▲13We built live AI evaluation to detect hallucinations, grounding issues, and drift as outputs are generated. This helps teams move GenAI from pilots into production.
2025 · ragmetrics.ai · its alternatives →
- 13
Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…
Jul 2026 · oodle.ai · its alternatives →
- 14
Hey HN, We’re two developers (co-founders) with a team of 20 who got tired of spending hours reviewing PRs, so we built Infinitcode.ai, an AI-powered code reviewer that: - *Summarizes PRs in plain English*: No more deciphering 1,000-line diff jungles - *Catches more than bugs*: Security holes, performance pitfalls, code smells, even typos (yes, we’ll flag “vurnerabilities” and vulnerabilities) - *Zero onboarding*: Works instantly—no “let me learn your codebase for weeks” nonsense. Why we’re posting: We’re in alpha and need brutal honesty. Roast our tool, mock our UI, or tell us why AI will…
2025 · infinitcode.ai · its alternatives →
- 15
Hi HN! I’m Paul, founder of Traceprompt. We’re building an open-source SDK that wraps your AI agents and generates tamper-proof audit trails, so you can prove what agent did what, when and with which model. You can check out the Node SDK (more languages coming soon): https://github.com/traceprompt/traceprompt-node We built Traceprompt because LLMs are being used everywhere without a clear plan for audit and compliance. From research and discussions with other devs, I often see fintech and healthcare teams manually stitching together API Gateway, CloudWatch Logs and S3…
2025 · traceprompt.com · its alternatives →
- 16AF
We’ve built an AI risk assessment tool designed specifically for GenAI/LLM applications. It's still early, but we’d love your feedback. Here’s what it does: 1. it performs comprehensive AI risk assessments by analyzing your codebase against different AI regulation/framework or even internal policies. It identifies potential issues and suggests fixes directly through one click PRs. 2. the first framework the platform supports is OWASP Top 10 for LLM Applications 2025, upcoming framework will be ISO 42001 as well as custom policy documents. 3. we're a small, early stage team, so the…
2025 · gettavo.com · its alternatives →
- 17
We are the developers of Phoenix, which we released in April of this year with a goal of bringing LLM observability to the notebook. In the time since, the growth of LLM frameworks and complex agent workflows led us to add support for LLM spans and traces and introduce a simple Eval harness for testing the data from those spans. The latest Traces & Spans release of Phoenix offers: -Out of the box tracing for LlamaIndex and LangChain -Fully local execution, no data sent anywhere, outside of your own LLM calls -Ability to get a common dataframe format across frameworks back to a notebook for…
2023 · github.com · its alternatives →
- 18
- 19
I’m a former Systems Architect (Cisco/VMware) turned builder in Thailand. TheAuditor v2.0 is a complete architectural rewrite (800+ commits) of the prototype I posted three months ago. The "A-ha" moment for me didn't come from a success; it came from a massive failure. I was trying to use AI to refactor a complex schema change (a foundation change from "Products" to "ProductsVariants"), and due to the scope of it, it failed spectacularly. I realized two things: * Context Collapse: The AI couldn't keep enough files in its context window to understand the full scope of the refactor, so it…
Dec 2025 · github.com · its alternatives →
- 20
Hi hacker news, My name is Dillion and I'm the creator of llm.report. A few months ago, I was frustrated by the lack of observability into the OpenAI API. All of us are left in the dark about API performance, latency, cost calculation, cost breakdown, and more. I just wanted to know more about how my AI app is performing in production and make data-driven decisions to improve the product. So I ended up just building it myself. There are three parts to the platform: 1. OpenAI API Dashboard (no-code) - Enter your OpenAI key and get access to detailed insights straight from the OpenAI API…
2023 · github.com · its alternatives →
- 21
Monitor every LLM API call and cost in real time
Mar 2026 · aicostguard.com · its alternatives →
- 22
I started using Claude Code (claude --dangerously-skip-permissions) and Codex (codex --yolo) and realized I had no reliable way to know what they actually did. The agent's own output tells you a story, but it's the agent's story. logira records exec, file, and network events at the OS level via eBPF, scoped per run. Events are saved locally in JSONL and SQLite. It ships with default detection rules for credential access, persistence changes, suspicious exec patterns, and more. Observe-only – it never blocks. https://github.com/melonattacker/logira
Mar 2026 · github.com · its alternatives →
- 23HW
A bunch of companies that I spoke to had their own claude & codex OTel dashboards that showed spend + seats per month. However, none of the dashboards actually analyzed how the engineers worked with the tools and if there were any areas for improvement! That's why I created https://www.promptster.ai. Managers get aggregate level view of code quality and how that ties with team workflows (nothing on a per-engineer level). While engineers get personalized coaching on how they can save tokens while keeping output high. We also have a tool built for individuals to test their local…
Jul 2026 · its alternatives →
- 24
AgentWatch▲12Hi HN, I’m a solo developer and built AgentWatch to solve a problem I kept running into while building AI agents: preventing runaway loops and unexpected LLM spend before requests reach the model. AgentWatch sits in front of OpenAI, Anthropic, Gemini, Bedrock, Azure OpenAI, Groq, and others to enforce budgets and runtime policies. I’d really appreciate your feedback. If you’re building AI agents, does this solve a problem you’ve experienced? I’d also love to hear what you’d improve or challenge.
Jun 2026 · agent-watch.dev · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →