AI · alternatives · 2026
24 alternatives to LLMtest
The pytest framework for testing LLM outputs
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
TensorZero▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
- 2
A pytest plugin for running and analyzing LLM evaluation tests. - AlmogBaku/pytest-evals
2025 · github.com · its alternatives →
- 3
- 4
How pytest, unittest, and nose deal with assertions.
2016 · pythontesting.net · its alternatives →
- 5
- 6
- 7
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
- 8
Spelltest framework simulates conversations between AI ‘synthetic users' in an environment to test and refine LLM-based applications. It ensures your app converse with utmost accuracy and relevance. Post-chat, Spelltest assesses responses, providing qualitative and quantitative feedback on performance. Suitable for both chat and completion modes. When to use: - After modifying your prompt. - When your LLM provider updates. - As a CI step for you repo. All feedback and collaborations appreciated!
2023 · github.com · its alternatives →
- 9
Hey HN, Anders and Tom here - we’ve been building an end-to-end testing framework powered by visual LLM agents to replace traditional web testing. We know there's a lot of noise about different browser agents. If you've tried any of them, you know they're slow, expensive, and inconsistent. That's why we built an agent specifically for running test cases and optimized it just for that: - Pure vision instead of error prone "set-of-marks" system (the colorful boxes you see in browser-use for example) - Use tiny VLM (Moondream) instead of OpenAI/Anthropic computer use for dramatically…
2025 · github.com · its alternatives →
- 10
Hi HN! We’re Adrien and Kanav. We met at our previous job, where we spent about a third of our lives combating a constant firehose of bugs. In the hope of reducing this pain for others in the future, we’re working on automating debugging. We’re currently working on a platform that ingests logs and then automatically reproduces, root causes and ultimately fixes production bugs as they happen. You can see some of our work on this here - https://news.ycombinator.com/item?id=39528087 As we were building the root-cause phase of our automated debugger, we realized that we developed…
2024 · github.com · its alternatives →
- 11
- 12
- 13
SelfHostLLM▲134A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
- 14
Ponicode CLI▲140Achieve high code quality with AI unit test generation
2021 · circleci.com · its alternatives →
- 15
Documentation for the Pydantic stack. Validate data with Pydantic Validation, build production AI agents with Pydantic AI, and monitor LLM calls, agent behavior, and costs with Pydantic Logfire.
2017 · pydantic-docs.helpmanual.io · its alternatives →
- 16
Build local LLMs using top data science libraries
2023 · its alternatives →
- 17
The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
2023 · github.com · its alternatives →
- 18
A pytest-based framework for testing multi AI agents systems
2025 · github.com · its alternatives →
- 19
- 20KA
I built this because Cursor, Claude Code and other agentic AI tools kept giving me tests that looked fine but failed when I ran them. Or worse - I'd ask the agent to run them and it would start looping: fix tests, those fail, then it starts "fixing" my code so tests pass, or just deletes assertions so they "pass". Out of that frustration I built KeelTest - a VS Code extension that generates pytest tests and executes them, got hooked and decided to push this project forward... When tests fail, it tries to figure out why: - Generation error: Attemps to fix it automatically, then tries again -…
Jan 2026 · keelcode.dev · its alternatives →
- 21
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
- 22
The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
2023 · github.com · its alternatives →
- 23
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com · its alternatives →
- 24
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →