Trying to fix the web scraping industry's benchmark problem
We've been trying to evaluate web scraping companies, but when you look at their benchmarks, you can't verify anything, and they mostly exist to prove the company is successful. They put somewhere between 98% and 100% because they pick their own urls, define success their own way, and don't publish the harness. We also saw companies like scrapfly astroturf websites like scrapeway and call them independent. So, we built an open source benchmark that we want to represent the frontier of web data. We're trying to look across all major anti-bot providers and industries, to build a comprehensive…
In plain words
This is an open-source benchmark tool designed to evaluate web scraping companies with transparent, verifiable testing. Unlike existing benchmarks that use proprietary criteria and hand-picked websites, this tool publishes its full test harness, target list, and pass criteria on GitHub for anyone to inspect and run. It tests scraping performance across major anti-bot providers and multiple industries, with monthly maintenance updates. The tool is built for developers and procurement teams who need honest performance comparisons rather than vendor-supplied claims.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
We've been trying to evaluate web scraping companies, but when you look at their benchmarks, you can't verify anything, and they mostly exist to prove the company is successful. They put somewhere between 98% and 100% because they pick their own urls, define success their own way, and don't publish the harness. We also saw companies like scrapfly astroturf websites like scrapeway and call them independent. So, we built an open source benchmark that we want to represent the frontier of web data. We're trying to look across all major anti-bot providers and industries, to build a comprehensive hard-target test set. It's all in the repo, you can run it, and we pledge to maintain it monthly. Curious what people think we could do to improve it? Would you change any of the conditions, add sites, etc. Our target list and pass criteria are something we're trying to improve, so would love to get feedback from the community on how we can actually provide something useful for people to test against!
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, July 2026
the whole month →- IR
I might be the only SRE on Earth with his own bowling center. It's a more in-depth gig than you'd think. My family and I bought an abandoned 8-lane bowling center in the rural mid-west. In our small town there weren't many recreation options for families. You've heard of a food desert? This is an R&R desert. It had been abandoned for a good reason. The roof leaks, the electrical system was constantly surging, and my 70-year-old bowling equipment (still) doesn't work perfectly. The system that keeps your score is particularly interesting to me. It's the thing you watch during your game, but…
Life & fun · Jul 2026
- EElevators▲1,680
Life & fun · Jul 2026 · john.fun
- 1W18 Words▲1,160
Life & fun · Jul 2026 · 18words.com
- BA
Over the past few months, our team has been building more and more slidedecks using web frontend technologies with coding harnesses like Claude Code, but a common complaint is to make even small edits we need to edit the code either manually or via the harness. To avoid this loop, I ended up creating Bento, a single HTML file with everything you need in a slide tool including animations and shared editing. There's no install or cloud login, everything works offline. The default deck is around 560 KB and it doesn't need to fetch anything once you got it. Open it in a browser and then you can…
Dev tools · Jul 2026 · bento.page
- GG
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…
AI · Jul 2026 · github.com
