nowfound

Dev tools · July 7, 2020

Crawlab

Web crawler admin platform for any framework or language

Crawlab launched on July 7, 2020 with 1 votes. The site was still online in September 2026, 6 years on.

Visit github.com ↗Crawlab alternatives

What it does

Crawlab is a Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java and various web crawler frameworks including Scrapy, Puppeteer, Selenium. It currently has 6k+ stars on Github.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 - crawlab-team/crawlab

Does a similar job

all alternatives →
  • Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language2019 · github.com · ▲116

    Distributed web crawler admin platform for spiders management regardless of languages and frameworks. - tikazyq/crawlab

  • Crawlee2022 · crawlee.dev · ▲229

    Build reliable web scrapers and robots, fast!

  • Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

    Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

  • Crawly2016 · crawly.diffbot.com · ▲606

    Never write another web scraper

  • Crawler.shMar 2026 · crawler.sh · ▲349

    Ultra-fast desktop & CLI app that finds your SEO & AEO issues and allows you to extract your content as Markdown

  • Headless Chrome Crawler2018 · github.com · ▲172

    Distributed crawler powered by Headless Chrome. Contribute to yujiosaka/headless-chrome-crawler development by creating an account on GitHub.

More dev tools this month

the category →
  • /dev/null▲489

    a social network for devs

    Dev tools · 3d ago · dev-null.sh

  • Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

    open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

    Dev tools · 28d ago · github.com

  • Reladraw – A diagram language where you decide where to place things▲411

    Contribute to reladraw/reladraw development by creating an account on GitHub.

    Dev tools · 26d ago · github.com

  • OpenBot▲362

    Grok Bot alternative: free, local, open-source, multiplayer

    Dev tools · 23d ago · openbot.run

  • Talanoa▲346

    Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

    Dev tools · 21d ago · talanoa.email

  • CoreSpeed▲332

    One MCP for everything your agents need: apps, memory, tools

    Dev tools · 29d ago · corespeed.io

Source: Product Hunt launch ↗

Launched alongside, July 2020

the whole month →