EvalCore
Snapshot-test AI behavior in CI
EvalCore launched on July 20, 2026 with 9 votes, #1,899 of 24,623 launches that month and more than 90% of that year's launches.
What it does
EvalCore is a single-binary test runner for LLM apps and agents. Define cases and scorers in YAML, run local targets on every PR, and replay model or judge calls offline for $0. Use REST or shell targets, baselines, trials, model comparisons, and OTel traces.
EvalCore is snapshot testing for AI behavior. Record how your LLM app or agent behaves, then replay it in CI on every change: offline, deterministic, $0.
Change a prompt, swap a model, update a dependency. Did anything break? EvalCore records how your AI behaves and checks every change against it. One small binary, any language, no flaky tests. $ evalcore run evals.yaml PASS late-refund (11ms) PASS fee-dispute (10ms) PASS card-lost (10ms) PASS wire-eta (9ms) 4 passed, 0 failed, 4 total GATE PASS pass_rate >= 0.95 (actual 1.00) ✔ PASSED One file. One live run. Replay forever. An eval is a YAML file plus a JSONL dataset. Run it once against the real model to record a cassette, then CI replays the recording on every change: offline, keyless, deterministic. No SDK, no test harness. Point a target at your app, list the dataset, stack scorers on…from evalcore.cc
Does a similar job
all alternatives →
- DADeltix – AI Driven TestingAug 2026 · app.deltix.ai · ▲54
Write a task in plain English. An AI agent runs it on a simulator on your Mac and tells you if a real user could complete it. Save the successful run as a regression check you can replay later.
Create LLM graders and run evals in JavaScript with one file2025 · github.com · ▲28Hi HN! Run it: OPENROUTER_API_KEY="sk" npx bff-eval --demo We built a tool to help people take LLM outputs and easily grade them / eval them to know how good an assistant response is. We've built a number of LLM apps, and while we could ship decent tech demos, we were disappointed with how they'd perform over time. We worked with a few companies who had the same problem, and found out scientifically building prompts and evals is far from a solved problem... writing these things feels more like directing a play than coding. Inspired by Anthropic's constitutional ai concepts, and amazing…
Prompt-Engineering Tool: AI-to-AI Testing for LLM2023 · github.com · ▲37Spelltest framework simulates conversations between AI ‘synthetic users' in an environment to test and refine LLM-based applications. It ensures your app converse with utmost accuracy and relevance. Post-chat, Spelltest assesses responses, providing qualitative and quantitative feedback on performance. Suitable for both chat and completion modes. When to use: - After modifying your prompt. - When your LLM provider updates. - As a CI step for you repo. All feedback and collaborations appreciated!
DeepEval – Evaluation and Unit Testing for LLMs2023 · github.com · ▲18The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Product Hunt launch ↗
Launched alongside, July 2026
the whole month →- IR
I might be the only SRE on Earth with his own bowling center. It's a more in-depth gig than you'd think. My family and I bought an abandoned 8-lane bowling center in the rural mid-west. In our small town there weren't many recreation options for families. You've heard of a food desert? This is an R&R desert. It had been abandoned for a good reason. The roof leaks, the electrical system was constantly surging, and my 70-year-old bowling equipment (still) doesn't work perfectly. The system that keeps your score is particularly interesting to me. It's the thing you watch during your game, but…
Life & fun · Jul 2026
18 Words▲1,160Play 18 Words, the free daily word puzzle. The official home is 18words.com. Tap Play now to open today
Life & fun · Jul 2026 · 18words.com
- BA
Over the past few months, our team has been building more and more slidedecks using web frontend technologies with coding harnesses like Claude Code, but a common complaint is to make even small edits we need to edit the code either manually or via the harness. To avoid this loop, I ended up creating Bento, a single HTML file with everything you need in a slide tool including animations and shared editing. There's no install or cloud login, everything works offline. The default deck is around 560 KB and it doesn't need to fetch anything once you got it. Open it in a browser and then you can…
Dev tools · Jul 2026 · bento.page
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…
AI · Jul 2026 · github.com
OpenSEO▲928Hello! For the last 2 months, I've been building OpenSEO. It's a user friendly UI wrapping the DataForSEO API which has high quality SEO data for keywords, backlinks, AI SEO and competitor research. Instead of paying $100+ per month, you buy DataForSEO credits and pay only for usage. The coolest thing about the project so far has been hearing about what people are building on top of it. People are using coding agents to quickly integrate with Google Search Console, Google Ads and building their own AI SEO and reporting features. Many were already building their own custom solutions around…
Dev tools · Jul 2026 · openseo.so