Alternatives
Products that do what Auditi does
Open source AI agents observability and evaluation
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
Trace, evaluate, and improve AI agents in production
Aug 2026 · telerik.com
- 10LO
Hi HN! Langfuse is OSS observability and analytics for LLM applications (repo: https://github.com/langfuse/langfuse, 2 min demo: https://langfuse.com/video, try it yourself: https://langfuse.com/demo) Langfuse makes capturing and viewing LLM calls (execution traces) a breeze. On top of this data, you can analyze the quality, cost and latency of LLM apps. When GPT-4 dropped, we started building LLM apps – a lot of them! [1, 2] But they all suffered from the same issue: it’s hard to assure quality in 100% of cases and even to have a clear view…
2023 · github.com
- 11
Audit your site for the AI search era. 100% Open Source
May 2026 · freeaiseoaudit.com
- 12
- 13YD
If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
2024 · github.com
- 14FF
I started leaning in on AI heavily this year, as I wanted to get more done autonomously, but then my token usage climbed dramatically to the point where my weekly quota would run out before the end of the week, sometimes a couple of days into the week. I realised I had to do something about it else I'd have to double my spend. So I decided to start tracking my cost per task type. This revealed that a lot of my spend went to searches/scans or simple things like scouting tasks. I then decided to turn this into a simple CLI tool that can be used to read your OpenAI-style logs locally, and…
Jul 2026 · github.com
- 15OA
Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…
Jul 2026 · oodle.ai
- 16
- 17WB
Hey HN, We’re two developers (co-founders) with a team of 20 who got tired of spending hours reviewing PRs, so we built Infinitcode.ai, an AI-powered code reviewer that: - *Summarizes PRs in plain English*: No more deciphering 1,000-line diff jungles - *Catches more than bugs*: Security holes, performance pitfalls, code smells, even typos (yes, we’ll flag “vurnerabilities” and vulnerabilities) - *Zero onboarding*: Works instantly—no “let me learn your codebase for weeks” nonsense. Why we’re posting: We’re in alpha and need brutal honesty. Roast our tool, mock our UI, or tell us why AI will…
2025 · infinitcode.ai
- 18LE
I started using Claude Code (claude --dangerously-skip-permissions) and Codex (codex --yolo) and realized I had no reliable way to know what they actually did. The agent's own output tells you a story, but it's the agent's story. logira records exec, file, and network events at the OS level via eBPF, scoped per run. Events are saved locally in JSONL and SQLite. It ships with default detection rules for credential access, persistence changes, suspicious exec patterns, and more. Observe-only – it never blocks. https://github.com/melonattacker/logira
Mar 2026 · github.com
- 19
- 20OS
Hi HN, Hugh and Vince here. LLMonitor helps you record, trace & search your LLM queries and chatbot conversations. You can also capture user feedback on your frontend and correlate it with backend LLM queries then use that to fine-tune your own models. The project started has an internal tool in our previous (failed) AI startup. We’re aware the LLM observability space is very crowded. Apart from being open-source, we differentiate with: - Model-agnostic and minimal lock-in (no MITM of requests). - High focus on DX and dashboard clarity. - Support for complex scenarios: e.g. a chatbot that…
2023 · github.com
- 21
- 22
Runtime governance for AI agents. Allow, warn, or block every model and tool call before it commits. Hash-chained audit for every decision. Compliance packs for SOC 2, HIPAA, PCI DSS, EU AI Act, SR 11-7, and FDA CSA. Apache 2.0. - sseshachala/conductai
9d ago · github.com
- 23
- 24MA
We built meta-agent: an open-source library that automatically and continuously improves agent harnesses from production traces. Point it at an existing agent, a stream of unlabeled production traces, and a small labeled holdout set. An LLM judge scores unlabeled production traces as they stream. A proposer reads failed traces and writes one targeted harness update at a time, such as changes to prompts, hooks, tools, or subagents. The update is kept only if it improves holdout accuracy. On tau-bench v3 airline, meta-agent improved holdout accuracy from 67% to 87%. We open-sourced meta-agent.…
Apr 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →