LLMtest
The pytest framework for testing LLM outputs
LLMtest launched on March 9, 2026.
What it does
Test your AI outputs like you test your code. Fast, Pydantic-based assertionLLMTest is an open-source pytest plugin that brings real software testing practices to AI applications. Write assertions for your LLM outputs the same way you write unit tests for your code. s for AI applications.
Test your AI outputs like you test your code. Fast, Pydantic-based assertions for AI applications.
from a s s e r t l l m import e x p e c t , l l m _ t e s t @llm_test ( e x p e c t . contains ( "Paris" ) , e x p e c t . latency_under ( 2000 ) , e x p e c t . cost_under ( 0.001 ) , model = "claude-sonnet-4-6" , ) def test_capital ( l l m ) : llm ( "What is the capital of France?" ) Terminal Waiting... Everything you need to test LLMs No LLM judge. No YAML configs. Just pytest. Most assertions are deterministic and instant. No paying an LLM to judge your output. All models use BaseModel — auto-validation, JSON serialization, schema generation. Text, performance, agent, and composable. Contains, regex, JSON, cost, latency, tool calls. OpenAI, Anthropic, Ollama out of the box. Install only…from assertllm.dev
Does a similar job
all alternatives →
TensorZero2025 · github.com · ▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
Pytest-evals – Simple LLM apps evaluation using pytest2025 · github.com · ▲13A pytest plugin for running and analyzing LLM evaluation tests. - AlmogBaku/pytest-evals
Python Test Podcast - pytest assert magic2016 · pythontesting.net · ▲52How pytest, unittest, and nose deal with assertions.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Product Hunt launch ↗
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
Littlebird▲709Littlebird remembers everything across your meetings, messages, docs, and apps, then takes action so you can recall any detail and never lose context.
AI · Mar 2026 · littlebird.ai
