nowfound

Life & fun · December 10, 2014

Common Crawl

Open repository of web crawl data

Common Crawl launched on December 10, 2014 with 95 votes, #350 of 1,837 launches that month and more than 50% of that year's launches. The site was still online in September 2026, 11 years on.

What it does

We build and maintain an open repository of web crawl data that can be accessed and analyzed by anyone.

Does a similar job

all alternatives →
  • Common Crawl2021 · ▲68

    Start a phrase and let the web complete it

  • Crawlee2022 · crawlee.dev · ▲229

    Build reliable web scrapers and robots, fast!

  • Webᵀ Crawl by Web Transpose2023 · ▲98

    Realtime web data for LLMs

  • Diggernaut2016 · diggernaut.com · ▲101

    Turn website content into dataset

  • searchcode2014 · searchcode.com · ▲75

    A source code search engine

  • CrawlBoard2017 · promptcloud.com · ▲10

    Access Clean, Free and Structured Data Feeds

More life & fun this month

the category →

Source: Product Hunt launch ↗

Launched alongside, December 2014

the whole month →
  • Noizio▲779

    Ambient background sounds for your Mac

    Work · 2014 · noizioapp.com

  • Use your iPad as a second display for your Mac

    Work · 2014 · duetdisplay.com

  • A simple way to bookmark & curate product collections

    Life & fun · 2014

  • Workflow▲556

    Automate things you do every day on your iPhone/iPad

    Dev tools · 2014 · apps.apple.com

  • Clear, effective, mistake-free writing everywhere you type.

    Work · 2014 · grammarly.com

  • Front 2.0▲500

    Shared team inboxes to manage Email, Twitter, SMS & more

    Work · 2014 · front.com