We benchmarked 18 LLMs on OCR (7K+ calls) – cheaper models win
Deploy AI Agents with confidence. AI Agent Evaluations done right.
We benchmarked 18 LLMs on OCR (7K+ calls) – cheaper models win launched on April 22, 2026 with 5 votes, #1,846 of 21,248 launches that month and more than 75% of that year's launches.
What it does
Stop overpaying for LLM calls by defaulting to flagship models. We ran 7,560 tests across 18 models (OpenAI, Anthropic, Google, Mistral) and found that mid-tier models often match state-of-the-art accuracy at 1/10th the cost. Arbitr lets you audit your own documents against 18+ LLMs side-by-side. Compare accuracy, cost, and reliability in real-time to find your perfect model fit. - Side-by-side OCR audit - Cost-per-success metrics - Open-source benchmark framework
Business-first OCR benchmark for standard operational documents. This benchmark compares OCR extraction performance on real business documents across repeated runs, so you can see both quality and consistency, not just a single score. It explicitly measures how well models transform an input document and expected keys into the correct output values. It highlights what matters in production: critical-field success, reliability over repeats (pass^n), latency, stability, and cost per successful outcome. Every model is run at its lowest thinking mode for a level playing field, unless noted otherwise (*) . † pass n metric: Probability of n consecutive successes in n runs (strict).…from arbitrhq.ai
Does a similar job
all alternatives →
- ISI spent 8 months trying to make LLMs Hack2025 · ▲6
Hey HN! For that last 8 months I've been trying to make agents that can hack web applications to find vulnerabilities in them - An AI Security Tester. The system has 29 agents in total, a custom LLM Orchestration framework which works on the task-subtask architecture (old-school but works amazingly for my use case, and is pretty reliable) with custom agent calling mechanism. No Auo-Gen, Langchain and Crew AI - Everything custom built for pentesting. Each test runs in an isolated Kali linux environment (on AWS Fargate), where the agents have full access to the environment to undertake any…
- AAAI agents run my one-person company on Gemini's free tier – $0/monthMar 2026 · ▲16
I'm a solo dev in Taiwan. I built 4 AI agents that handle content, sales leads, security scanning, and ops for my tech agency — all on Gemini 2.5 Flash free tier (1,500 req/day). I use ~105. Monthly LLM cost: $0. Architecture: 4 agents on OpenClaw (open source), running on WSL2 at home with 25 systemd timers. What they do every day: - Generate 8 social posts across platforms (quality-gated: generate → self-review → rewrite if score < 7/10) - Engage with community posts and auto-reply to comments (context-aware, max 2 rounds) - Research via RSS + HN API + Jina Reader → feed…
I implemented evals metrics for LLMs that runs locally on your machine2023 · github.com · ▲22The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗
Launched alongside, April 2026
the whole month →
- AG
Thought the resources for GPU arch were lacking, so here we are
Life & fun · Apr 2026 · jaso1024.com
Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
AI · Apr 2026 · github.com
I made a laptop stand out of solid concrete, in the style of brutalist architecture, with themes of urbex and urban decay
Life & fun · Apr 2026 · sam-burns.com
With social media and now AI, its important to keep the indie web alive. There are many people who write frequently. Blogosphere tries to highlight them by fetching the recent posts from personal blogs across many categories. There are two versions: Minimal (HN-inspired, fast, static): https://text.blogosphere.app/ Non-minimal: https://blogosphere.app/ If you don't find your blog (or your favorite ones), please add them. I will review and approve it.
AI · Apr 2026 · text.blogosphere.app