Scorecard – Evaluate LLMs like Waymo simulates cars
Hey HN! I built self-driving sim and eval at Waymo. Now I’m building Scorecard to bring that approach to agent eval: reproducible, automated scoring for AI. Scorecard lets you: - Run LLM-as-judge evals on agent workflows: test tool usage, multi-step reasoning, and task completion in CI/CD or in a playground. - Debug failures with OpenTelemetry traces: see which tool failed, why your agent looped, and where reasoning went wrong. - Collaborate on datasets, simulated agents, and evaluation metrics. Try it out → https://app.scorecard.io (free tier, no payment required!) Docs →…
Scorecard – Evaluate LLMs like Waymo simulates cars launched on October 14, 2025 with 7 votes, #2,779 of 8,178 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
The simulation platform for building frontier AI agents
In the maker’s words, at launch
Hey HN! I built self-driving sim and eval at Waymo. Now I’m building Scorecard to bring that approach to agent eval: reproducible, automated scoring for AI. Scorecard lets you: - Run LLM-as-judge evals on agent workflows: test tool usage, multi-step reasoning, and task completion in CI/CD or in a playground.…
Does a similar job
all alternatives →
Agent-skills-eval – Test whether Agent Skills improve outputsMay 2026 · github.com · ▲79A test runner for agentskills.io-style AI agent skills - darkrishabh/agent-skills-eval
Create LLM graders and run evals in JavaScript with one file2025 · github.com · ▲28Hi HN! Run it: OPENROUTER_API_KEY="sk" npx bff-eval --demo We built a tool to help people take LLM outputs and easily grade them / eval them to know how good an assistant response is. We've built a number of LLM apps, and while we could ship decent tech demos, we were disappointed with how they'd perform over time. We worked with a few companies who had the same problem, and found out scientifically building prompts and evals is far from a solved problem... writing these things feels more like directing a play than coding. Inspired by Anthropic's constitutional ai concepts, and amazing…
Chunk sidecarsMay 2026 · circleci.com · ▲77Hi HN! My name is Olaf, I work at CircleCI as a technology advisor in the CTO office, came in through the acquisition of my company Vamp.io (progressive delivery for microservices on k8s) in 2021. Wanted to hear the HN community feedback and thoughts on a project we think could be very interesting when adding AI coding agents to the SDLC and your CI pipelines. Our team at CircleCI built Chunk sidecars after repeatedly running into the same issue internally: by the time our CI catches a failure, the agent has already moved on and most of the useful context is gone. The basic idea of Chunk…
I implemented evals metrics for LLMs that runs locally on your machine2023 · github.com · ▲22The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, October 2025
the whole month →
I went down the rabbit hole on a side project and ended up building this: Strange Attractors(https://blog.shashanktomar.com/posts/strange-attractors). It’s built with three.js. Working on it reminded me of the little "maths for fun" exercises I used to do while learning programming in early days. Just trying things out, getting fascinated and geeky, and being surprised by the results. I spent way too much time on this, but it was extreme fun. My favorite part: someone pointed me to the Simone Attractor on Threads. It is a 2D attractor and I asked GPT to extrapolate it to…
AI · Oct 2025 · blog.shashanktomar.com
Autism Simulator▲779Hey all, I built this. It’s not trying to capture every autistic experience (that’d be impossible). It’s based on my own lived experience as well as that of friends on the spectrum. I'm trying to give people a feel for what masking, decision fatigue, and burnout can look like day-to-day. That’s hard to explain in words, but easier to show through choices and stats. I'm not trying to "define autism". I’ve gotten good feedback here about resilience, meds, and difficulty tuning. I’ll keep tweaking it. If even a few people walk away thinking, "ah, maybe that’s why my coworker struggles in those…
Life & fun · 2025 · autism-simulator.vercel.app
