AI · alternatives · 2026
24 alternatives to EvalCore
Snapshot-test AI behavior in CI
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
APIEval-20▲121An open benchmark for AI agents that test APIs
May 2026 · resources.kusho.ai · its alternatives →
- 2
- 3
Ito▲449I'm Evan and I made Ito.ai It's code review that actually runs your code. The result is that it finds more bugs with a smaller false positive rate. You get screenshots, videos, and run logs. I built Ito because manual verification sucks and I thought I could automate it. I've been a professional developer for 15 years and now more than ever, I'm just spending time making sure things work. I looked into all of the other agentic testing tools that click around a url you give them and frankly, they sucked. They just didn't work. I figured to be able to actually testing things you need to run…
Aug 2026 · ito.ai · its alternatives →
- 4
QualGent▲296I built an AI agent that can do mobile app QA after working at Google for years and waiting weeks for QA results. It runs on real devices and uses an ensemble of agents (including computer use) to test like a human. Happy to answer questions and hope it helps app devs ship faster without bugs!
Dec 2025 · qualgent.ai · its alternatives →
- 5DA
Write a task in plain English. An AI agent runs it on a simulator on your Mac and tells you if a real user could complete it. Save the successful run as a regression check you can replay later.
Aug 2026 · app.deltix.ai · its alternatives →
- 6
Hi HN! Run it: OPENROUTER_API_KEY="sk" npx bff-eval --demo We built a tool to help people take LLM outputs and easily grade them / eval them to know how good an assistant response is. We've built a number of LLM apps, and while we could ship decent tech demos, we were disappointed with how they'd perform over time. We worked with a few companies who had the same problem, and found out scientifically building prompts and evals is far from a solved problem... writing these things feels more like directing a play than coding. Inspired by Anthropic's constitutional ai concepts, and amazing…
2025 · github.com · its alternatives →
- 7
I use AI agents to build UI features daily. The thing that kept annoying me: the agent writes code but never sees what it actually looks like in the browser. It can’t tell if the layout is broken or if the console is throwing errors. So I built a CLI that lets the agent open a browser, interact with the page, record what happens, and collect any errors. Then it bundles everything — video, screenshots, logs — into a self-contained HTML file I can review in seconds. proofshot start --run "npm run dev" --port 3000 # agent navigates, clicks, takes screenshots proofshot stop It works with…
Mar 2026 · github.com · its alternatives →
- 8
- 9
Hey HN, Hakim here from Fini (YC S22), a startup focused on providing automated customer support bots for enterprises that have a high volume of support requests. Today, one of the largest use cases of LLMs is for the purpose of automating support. As the space has evolved over the past year, there has subsequently been a need for evaluations of LLM outputs - and a sea of LLM Evals packages have been released. "LLM evals" refer to the evaluation of large language models, assessing how well these AI systems understand and generate human-like text. These packages have recently relied on…
2024 · github.com · its alternatives →
- 10
A test runner for agentskills.io-style AI agent skills - darkrishabh/agent-skills-eval
May 2026 · github.com · its alternatives →
- 11
Spelltest framework simulates conversations between AI ‘synthetic users' in an environment to test and refine LLM-based applications. It ensures your app converse with utmost accuracy and relevance. Post-chat, Spelltest assesses responses, providing qualitative and quantitative feedback on performance. Suitable for both chat and completion modes. When to use: - After modifying your prompt. - When your LLM provider updates. - As a CI step for you repo. All feedback and collaborations appreciated!
2023 · github.com · its alternatives →
- 12
Cipherra▲5Infrastructure for Continuous Evals of AI Agents
May 2026 · cipherra.ai · its alternatives →
- 13
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
2023 · vectara.com · its alternatives →
- 14
The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
2023 · github.com · its alternatives →
- 15BE
Hey HN, We're excited to introduce Braintrust, a platform for running and tracking AI evaluations (“evals”) [1]. At my previous startup Impira and leading AI at Figma, we had this recurring problem where we never knew if changes we made to our products would improve or regress key user scenarios. We built some tooling to solve this problem and after talking to other developers learned that it was a widespread issue. Specifically, it’s challenging to establish a great dev loop that lets you systematically improve and ship high quality AI products. We worked with the teams at Zapier, Coda, and…
2023 · its alternatives →
- 16
The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
2023 · github.com · its alternatives →
- 17IF
The point of this seemingly useless exercise was to stress-test my elisp-eval MCP https://github.com/agzam/death-contraptions/tree/main/tools/... AI hands it a string of Lisp, it runs inside live Emacs session, poking into it via emacsclient. Behaves like a REPL the LLM can drive - state persists between the calls. Emacs is a Lisp machine. Buffers, files, network, subprocesses, games, major modes, the UI itself: every feature is already an elisp function. Hand an LLM a way to run arbitrary elisp and it inherits the entire Emacs API for free. No…
Apr 2026 · imgur.com · its alternatives →
- 18
DevLens▲8The Autonomous Intelligence Layer for Pull Requests
Apr 2026 · devlens.xyz · its alternatives →
- 19
Lovelaice▲10Multi-model AI testing, evaluation, optimization made simple
2025 · lovelaice.com · its alternatives →
- 20
Spec27▲34Hi HN! We’re a team of ML validation specialists and we’ve been building /Spec27, a tool for testing whether AI agents still do their job safely and reliably as models, prompts, tools, and surrounding systems change. We started working on this because a lot of current LLM evaluation work seems aimed at scoring general model behavior, while many teams are deploying systems that have a specific mission to fulfill. Many of the tools also assume you have full access to the agent stack and traces so you can place SDKs and Gateways, but a lot of agents are being created on vendor platforms…
Apr 2026 · spec27.ai · its alternatives →
- 21
RagMetrics▲13We built live AI evaluation to detect hallucinations, grounding issues, and drift as outputs are generated. This helps teams move GenAI from pilots into production.
2025 · ragmetrics.ai · its alternatives →
- 22
Hi HN, A few weeks ago, we stealth launched SEC Insights on Product Hunt and were thrilled to reach the top 5! What is SEC Insights? It harnesses the Retrieval Augmented Generation (RAG) capabilities of LlamaIndex to provide answers about SEC 10-K & 10-Q documents. The enthusiastic reception we received on Product Hunt prompted us to open-source our project. Now, developers can use our project either as a reference or they can fork it entirely for their LLM-powered applications. We at LlamaIndex built this to help developers move their LLM app ideas from prototype to production. We're very…
2023 · github.com · its alternatives →
- 23MF
2024 · mutahunter.ai · its alternatives →
- 24
Hi HN, I've been working with LLMs in production for a while both as a solo dev building apps for clients and working at an AI startup. The one thing that always was a pain was to pay OpenAI/Gemini/Anthropic a few dollars a month just for me to say "test" or have a CI runner validate some UI code. So I built this server called ChunkBack, that mocks the popular llm provider's functionality but allows you to type in a deterministic language: `SAY "cheese"` or `TOOLCALL "tool_name" {} "tool response"` I've had to work in some test environments and give good results for experimenting…
Nov 2025 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →