Alternatives
Products that do what LLMtest does
The pytest framework for testing LLM outputs
- 1PE
2025 · github.com
- 2
- 3
- 4TV
May 2026 · github.com
- 5
- 6
- 7
- 8
- 9PE
Spelltest framework simulates conversations between AI ‘synthetic users' in an environment to test and refine LLM-based applications. It ensures your app converse with utmost accuracy and relevance. Post-chat, Spelltest assesses responses, providing qualitative and quantitative feedback on performance. Suitable for both chat and completion modes. When to use: - After modifying your prompt. - When your LLM provider updates. - As a CI step for you repo. All feedback and collaborations appreciated!
2023 · github.com
- 10MO
Hey HN, Anders and Tom here - we’ve been building an end-to-end testing framework powered by visual LLM agents to replace traditional web testing. We know there's a lot of noise about different browser agents. If you've tried any of them, you know they're slow, expensive, and inconsistent. That's why we built an agent specifically for running test cases and optimized it just for that: - Pure vision instead of error prone "set-of-marks" system (the colorful boxes you see in browser-use for example) - Use tiny VLM (Moondream) instead of OpenAI/Anthropic computer use for dramatically…
2025 · github.com
- 11LD
Hi HN! We’re Adrien and Kanav. We met at our previous job, where we spent about a third of our lives combating a constant firehose of bugs. In the hope of reducing this pain for others in the future, we’re working on automating debugging. We’re currently working on a platform that ingests logs and then automatically reproduces, root causes and ultimately fixes production bugs as they happen. You can see some of our work on this here - https://news.ycombinator.com/item?id=39528087 As we were building the root-cause phase of our automated debugger, we realized that we developed…
2024 · github.com
- 12
- 13
- 14
- 15
- 16KA
I built this because Cursor, Claude Code and other agentic AI tools kept giving me tests that looked fine but failed when I ran them. Or worse - I'd ask the agent to run them and it would start looping: fix tests, those fail, then it starts "fixing" my code so tests pass, or just deletes assertions so they "pass". Out of that frustration I built KeelTest - a VS Code extension that generates pytest tests and executes them, got hooked and decided to push this project forward... When tests fail, it tries to figure out why: - Generation error: Attemps to fix it automatically, then tries again -…
Jan 2026 · keelcode.dev
- 17LA
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com
- 18AT
2017 · github.com
- 19TO
Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2024 · github.com
- 20DE
2023 · github.com
- 21LS
Hi, I was a corporate lawyer for many years working with a lot of financial services and insurance companies. In practicing law, I noticed there was a lot of repetition in the tasks I was working on even as a highly paid attorney that could be automated. I wanted to solve the problem of dealing with a lot information and data in a practical way, using AI. This motivated me to start AI Bloks/LLMWare with my husband, who had a deep background in software and is a very early adopter of AI. We have been on this journey with our open source project LLMWare for the past 4 months, producing a…
2024 · github.com
- 22LT
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com
- 23
- 24AE
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →