Benchmarking VLMs vs. Traditional OCR
Vision models have been gaining popularity as a replacement for traditional OCR. Especially with Gemini 2.0 becoming cost competitive with the cloud platforms. We've been continuously evaluating different models since we released the Zerox package last year (https://github.com/getomni-ai/zerox). And we wanted to put some numbers behind it. So we’re open sourcing our internal OCR benchmark + evaluation datasets. Full writeup + data explorer here: https://getomni.ai/ocr-benchmark Github: https://github.com/getomni-ai/benchmark Huggingface:…
Benchmarking VLMs vs. Traditional OCR launched on February 20, 2025 with 146 votes, #274 of 5,999 launches that month and more than 95% of that year's launches. The site was not reachable when checked in September 2026.
In plain words
An open-source OCR benchmark and evaluation dataset that compares vision language models against traditional OCR systems. Created for developers and researchers evaluating optical character recognition tools, it measures model performance using JSON accuracy to assess how well different OCR providers prepare data for large language model processing. The benchmark includes real evaluation datasets and a data explorer, supporting the comparison of newer vision models like Gemini 2.0 with established OCR platforms.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Vision models have been gaining popularity as a replacement for traditional OCR. Especially with Gemini 2.0 becoming cost competitive with the cloud platforms. We've been continuously evaluating different models since we released the Zerox package last year (https://github.com/getomni-ai/zerox).…
Does a similar job
all alternatives →
Qwen-2.5-32B is now the best open source OCR model2025 · github.com · ▲211Last week was big for open source LLMs. We got: - Qwen 2.5 VL (72b and 32b) - Gemma-3 (27b) - DeepSeek-v3-0324 And a couple weeks ago we got the new mistral-ocr model. We updated our OCR benchmark to include the new models. We evaluated 1,000 documents for JSON extraction accuracy. Major takeaways: - Qwen 2.5 VL (72b and 32b) are by far the most impressive. Both landed right around 75% accuracy (equivalent to GPT-4o’s performance). Qwen 72b was only 0.4% above 32b. Within the margin of error. - Both Qwen models passed mistral-ocr (72.2%), which is specifically trained for OCR. - Gemma-3…
OCR Arena – A playground for OCR modelsNov 2025 · ocrarena.ai · ▲216I built OCR Arena as a free playground for the community to compare leading foundation VLMs and open-source OCR models side-by-side. Upload any doc, measure accuracy, and (optionally) vote for the models on a public leaderboard. It currently has Gemini 3, dots.ocr, DeepSeek, GPT5, olmOCR 2, Qwen, and a few others. If there's any others you'd like included, let me know!
OCR Benchmark Focusing on Automation2025 · nanonets.com · ▲58OCR/Document extraction field has seen lot of action recently with releases like Mixtral OCR, Andrew Ng's agentic document processing etc. Also there are several benchmarks for OCR, however all testing for something slightly different which make good comparison of models very hard. To give an example, some models like mixtral-ocr only try to convert a document to markdown format. You have to use another LLM on top of it to get the final result. Some VLM’s directly give structured information like key fields from documents like invoices, but you have to either add business rules on top…
Zerox – Document OCR with GPT-mini2024 · github.com · ▲246This started out as a weekend hack with gpt-4-mini, using the very basic strategy of "just ask the ai to ocr the document". But this turned out to be better performing than our current implementation of Unstructured/Textract. At pretty much the same cost. I've tested almost every variant of document OCR over the past year, especially trying things like table / chart extraction. I've found the rules based extraction has always been lacking. Documents are meant to be a visual representation after all. With weird layouts, tables, charts, etc. Using a vision model just make sense! In…
Build an open-source computer vision model in seconds using text2023 · usezeroshot.com · ▲64Hello HN! I want to share something me and a few friends have been working on for a while now — Zeroshot, a web tool that builds image classifiers using text-image models and autolabeling. What does this mean in practice? You can put together an image classifier in about 30 seconds that’s faster and more accurate than CLIP, but that you can deploy yourself however you’d like. It’s open source, commercially licensed, and doesn’t require you to pay anyone per API call. Here's a 2 minute video that shows it off: https://www.youtube.com/watch?v=S4R1gtmM-Lo How/why does it…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 28d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, February 2025
the whole month →
Screen Studio 3.0▲1,832Beautiful screen recordings with instant shareable links
Growth · 2025 · screen.studio
- IG
I was at FB/Meta from late 2013 to early 2023, mostly working in the compiler/runtime spaces. I got hit in the spring 2023 layoff wave. I immediately started making games in my newfound free time (a lifelong interest, and I even worked in AA(A?) back ca. ~2000), and in October 2023 I stumbled upon the idea of a roguelike pachinko/plinko game inspired by Luck Be A Landlord. Things snowballed quickly, I started talking to publishers, then worked like crazy through all of 2024, almost the hardest I've ever worked in my career, and launched the game in December 2024. It's sold…
Work · 2025
i wanted to change the habit of reaching for my phone in the morning and doomscrolling away an hour so i built an app to help me. now i have to literally touch grass before accessing my most distracting apps the app is built in swiftui, uses the screen time apis provided by apple and google vision to recognise grass or not i'd love to get your thoughts on the concept.
Life & fun · 2025 · touchgrass.now