Alternatives

Products that do what Common Crawl does

Open repository of web crawl data

1CO
Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language▲116
2019 · github.com
2
Pocket Explore▲351
A human curated version of the web
2016
3
Crawlee▲229
Build reliable web scrapers and robots, fast!
2022
4
Diggernaut▲101
Turn website content into dataset
2016
5
Online Web Scraper▲260
Cloud hosted data extraction tool
2016
6
searchcode▲75
A source code search engine
2014
7
Fosshub▲79
An open source for free and open source tools
2015
8VW
Voyager – write a web crawler/scraper as a state machine in Rust▲110
2020 · github.com
9
Ecommerce Stack▲152
A curated directory of E-commerce tools & contents
2016
10AT
A Twitter crawler for app ideas▲272
2021 · iwishtherewas.app
11
Leap.it▲76
A visual search engine for the web
2014
12
Portia▲349
Scrape websites visually
2016
13AO
An open distributed search engine for science▲98
2014 · juretriglav.si
14
Silk▲87
Data-driven website builder
2014
15
HyperCrawl ▲150
Super-fast web crawling for LLM development
2024
16
The Research Browser▲87
A powerful browsing application to collect and share info
2014
17
Prerender▲72
Crawlable HTML snapshots for SPAs
2014
18
SEM Stash▲173
100+ curated articles/resources for search engine marketers
2017
19
Crawlify▲165
AI powered data extraction APIs. Hassle-free data retrieval.
2020
20
Sidenotes▲125
Annotate the web.
2014
21
Datamatic▲156
Editable data visualizations for the web
2017
22
CODY▲136
Free library of HTML, CSS, & JS nuggets
2014
23
The evolution of the web▲94
A data visualization about the growth of the web since 1990
2015
24
Omniref▲144
Stackoverflow for Github
2014

Ranked by how close each launch is in meaning, then by votes. Refine with a description →