Crawlee

What it does

Crawlee is an intuitive, customizable open-source library for web scraping and browser automation. Quickly scrape data, store it, and avoid getting blocked with auto-generated human-like fingerprints, headless browsers, and smart proxy rotation.

• CW
  Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

  Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

• CO
  Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language2019 · github.com · ▲116
• 
  Spider Free2019 · ▲362

  Turn websites into organized data without code.

• 
  Crawl AI2025 · ▲116

  Build Your Own AI With One Prompt

• 
  Scraper AI2020 · ▲190

  Automate website data extraction in a few clicks

• 
  ScrapingBee (formerly ScrapingNinja)2019 · ▲275

  The easiest web scraping API on the web

• 
  Dograh▲592

  The open source VAPI alternative

  Dev tools · 25d ago · dograh.com

• 
  Meridian▲530

  Don't let your work go unnoticed. Get promoted!

  Dev tools · 20d ago · meridiona.com

• 
  x1▲516

  Lovable for iPhone apps go from idea to App Store

  Dev tools · 11d ago · x1.new

• 
  The GTM Co-Founder▲402

  Open-source GTM skills for technical founders

  Dev tools · 29d ago · gtmcofounder.com

• 
  Nuphos▲380

  The AI-Native DevOps Workspace.

  Dev tools · 24d ago · nuphos.ai

• 
  Lettertrace▲375

  Track your AI visibility for free (using your own API keys!)

  Dev tools · 25d ago · lettertrace.com

• 
  Xnapper▲1,517

  Take beautiful screenshots instantly

  Work · 2022 · xnapper.com

• 
  Gamma▲1,169

  Write like a doc, present like a deck

  Work · 2022 · gamma.app

• 
  Threado▲923

  The command center for community builders

  Life & fun · 2022

• 
  Remotebase▲742

  Build remote engineering teams in 24 hours

  Dev tools · 2022 · remotebase.com

• 
  DhiWise 2.0▲737

  Convert Figma designs into Flutter 3.0 & React code rapidly

  Dev tools · 2022 · dhiwise.com

• PA
  Pornpen.ai – AI-Generated Porn▲711

  Hey HN, I've been working on https://pornpen.ai, a site for generating adult images. Please only visit the site if you are 18+ and willing to look at NSFW images. This site is an experiment using newer text-to-image models. I explicitly removed the ability to specify custom text to avoid harmful imagery from being generated. New tags will be added once the prompt-engineering algorithm is fine-tuned further. If the servers are overloaded, take a look at the feed and search pages to look through past results. For comments/suggestions/feedback please visit…

  AI · 2022 · pornpen.ai