Alternatives
Products that do what AI-Evals.io – Evaluate this site with the tools it reviews does
I've been working on a site [1] to give people control of their LLM workflows through AI evals - automated checks that, once defined, let you move fast without regressions and cut through hype with proof. That one-liner is aimed at software engineers, but I've spent my career helping cross-functional teams collaborate, and that's really what this is about. AI agents make powerful workflows very plausible, but only if teams can grow them incrementally without losing control - no vendor lock-in, no discipline silos, no blind trust in outputs. The site tries to meet different audiences where…
- 1
- 2AS
May 2026 · github.com
- 3CL
Hi HN! Run it: OPENROUTER_API_KEY="sk" npx bff-eval --demo We built a tool to help people take LLM outputs and easily grade them / eval them to know how good an assistant response is. We've built a number of LLM apps, and while we could ship decent tech demos, we were disappointed with how they'd perform over time. We worked with a few companies who had the same problem, and found out scientifically building prompts and evals is far from a solved problem... writing these things feels more like directing a play than coding. Inspired by Anthropic's constitutional ai concepts, and amazing…
2025 · github.com
- 4
- 5
- 6
- 7WE
Hey HN! We’ve been building an MCP server to help AI-assisted web app developers by using browser agents to test whether changes made by an AI inside an editor actually work. We've been testing it on scenarios like verifying new flows in a UI, or checking that sending a chat request triggers a response. The idea is to let your coding agent both code and evaluate if what it did was correct. Here’s a short demo with Cursor: https://www.youtube.com/watch?v=_AoQK-bwR0w When building apps, we found the hardest part of AI-assisted coding isn’t the coding—it’s tedious point-and-click…
2025 · github.com
- 8MA
I've been exploring the (not so=) amazing potential of AI in coding and have compiled a list of tools. From AI-powered IDEs to code generators, this resource is my contribution to the community. I'm still on the fence about including txt2sql projects, as their functionality seems too basic to me. And I'm personally maintaining this, so your feedback is wellcome.
2025 · aicode.danvoronov.com
- 9
- 10AE
I’ve spent the past 10 years working on AI in finance, with much of that time focused on building evaluation systems for production environments. As agents become more widely adopted, more software engineering and product people have start building them. But I’ve noticed that many teams are not yet fluent in systematic evaluation, or in the processes needed to keep agent quality high over time. For large organizations, that gap is rarely the bottleneck due to dedicated teams. But after speaking with a number of startups, it became clear that building strong, up-to-date evals is much harder…
May 2026 · github.com
- 11SE
Hey HN! I built self-driving sim and eval at Waymo. Now I’m building Scorecard to bring that approach to agent eval: reproducible, automated scoring for AI. Scorecard lets you: - Run LLM-as-judge evals on agent workflows: test tool usage, multi-step reasoning, and task completion in CI/CD or in a playground. - Debug failures with OpenTelemetry traces: see which tool failed, why your agent looped, and where reasoning went wrong. - Collaborate on datasets, simulated agents, and evaluation metrics. Try it out → https://app.scorecard.io (free tier, no payment required!) Docs →…
Oct 2025 · docs.scorecard.io
- 12AB
Hi there, HN! We’re Jai and Sanket from DeepSource (YC W20), and today we’re launching Autofix Bot, a hybrid static analysis + AI agent purpose-built for in-the-loop use with AI coding agents. AI coding agents have made code generation nearly free, and they’ve shifted the bottleneck to code review. Static-only analysis with a fixed set of checkers isn’t enough. LLM-only review has several limitations: non-deterministic across runs, low recall on security issues, expensive at scale, and a tendency to get ‘distracted’. We spent the last 6 years building a deterministic, static-analysis-only…
Dec 2025
- 13
- 14
Deterministic offline release evidence for AI agents
Jul 2026 · iisacc-justmoong.github.io
- 15AF
We’ve built an AI risk assessment tool designed specifically for GenAI/LLM applications. It's still early, but we’d love your feedback. Here’s what it does: 1. it performs comprehensive AI risk assessments by analyzing your codebase against different AI regulation/framework or even internal policies. It identifies potential issues and suggests fixes directly through one click PRs. 2. the first framework the platform supports is OWASP Top 10 for LLM Applications 2025, upcoming framework will be ISO 42001 as well as custom policy documents. 3. we're a small, early stage team, so the…
2025 · gettavo.com
- 16II
2023 · github.com
- 17CO
Hi HN! I’m excited to share ControlFlow, our new open-source framework for building agentic workflows. ControlFlow is built around a core opinion that LLMs perform really well on small, well-defined tasks and run off the rails otherwise. I know that may seem obvious, but the key insight is that if you compose enough of these small tasks into a structured workflow, you can recover the kind of complex behaviors we associate with autonomous AIs, without sacrificing control or observability at each step. It ends up feeling a lot like writing a traditional software workflow. With ControlFlow you:…
2024 · github.com
- 18
- 19WW
Hey all! @sridatta and I wrote a book/zine called Forest Friends on system evals for LLM-driven apps. But it's a bit more whimsical, a bit more visual, and very much inspired by the meme of LLMs being a shoggoth polished into a smiley face with RLHF. LLM system evals are important as companies move past the flashy AI demos to reliable production apps. System evals keep coming up as the answer for what you "should do", but it's not exactly a standard part of the software engineering toolkit. So we pulled from @sridatta's seven years as a research engineer at Google, plus a ton of best…
2024
- 20IB
I built a tool to roast landing pages with AI agents. I was gathering feedback from watching landing page roast videos, and figured out I could prompt LLMs to analyse a screenshot and roast based on the same criteria. It's not 100% accurate yet, but it has been really insightful when I've tested it on my own websites. Let me know what you think!
2024 · roastmylandingpage.io
- 21BE
Hey HN, We're excited to introduce Braintrust, a platform for running and tracking AI evaluations (“evals”) [1]. At my previous startup Impira and leading AI at Figma, we had this recurring problem where we never knew if changes we made to our products would improve or regress key user scenarios. We built some tooling to solve this problem and after talking to other developers learned that it was a widespread issue. Specifically, it’s challenging to establish a great dev loop that lets you systematically improve and ship high quality AI products. We worked with the teams at Zapier, Coda, and…
2023
- 22PC
Hey HN — 2 months ago we shared our first product with the HN community (https://news.ycombinator.com/item?id=43362535). Despite receiving lots of traffic from HN, we didn’t see any traction or retention. One of our major takeaways was that our product was too complicated. So we’ve spent the last 2 months iterating towards a much more focused product that tries to do just one thing really well. Today, we’d like to share our second launch with HN. Our original idea was to help software engineers build high-quality LLM applications by integrating their domain knowledge into a…
2025 · withpi.ai
- 23
I run a small AI lab and playground and got super excited about Anthropics paper "Verbalizable Representations Form a Global Workspace in Language Models" (https://transformer-circuits.pub/2026/workspace/index.html) It talks about how they use a tool they call a Jacobian Lens to view inside the middle layers of LLM while it's working before it commits to a word (token). I wanted to see if I could get a version of this running on the open models and to my surprise it worked! I ran some experiments with it and build a public facing free tool anyone can use with your…
Jul 2026 · lucid.earthpilot.ai
- 24OS
Hey HN! We built EvalKit, a library you embed to capture agent actions and a UI where domain experts give feedback, evaluate and improve AI agents. We experienced, in large agentic systems, prompt-engineering or auto-prompt improvement tool can get accuracy from 0 to 50% but for increasing accuracy to 100% we had to work with domain experts. Example -> In a law ai agent, lawyers are needed because law is complex and lawyers have a deeper context compared to non-lawyers. Other evaluation tools in the market focus on the experience of the developer and we are focusing on making as easy as…
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →