nowfound

AI · September 25, 2018

Apify SDK – A scalable web crawling and scraping library for JavaScript

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headf

Apify SDK – A scalable web crawling and scraping library for JavaScript launched on September 25, 2018 with 78 votes, #513 of 2,635 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 8 years on.

Visit github.com ↗Alternativestop 19% of September 2018

In plain words

Apify SDK is a JavaScript library for building web crawlers and scraping applications at scale. Designed for developers, it provides tools to extract data from websites and automate web interactions. The library handles the technical complexities of crawling, allowing users to focus on defining what data to collect and how to process it. Available on GitHub, it has been in development since 2018.

written from the facts on this page · September 2026

From the sources

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation. - apify/crawlee

Does a similar job

all alternatives →
  • Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

    Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

  • Crawlee2022 · crawlee.dev · ▲229

    Build reliable web scrapers and robots, fast!

  • Apify2020 · apify.com · ▲271

    Thousands of tools to automate your business. Get real-time web data, track competitors, generate leads, and integrate your apps and AI agents.

  • Apifier – hosted web crawler for developers2015 · apifier.com · ▲101

    Thousands of tools to automate your business. Get real-time web data, track competitors, generate leads, and integrate your apps and AI agents.

  • Apifier2015 · apify.com · ▲223

    The web scraping and automation platform

  • Crawly2016 · crawly.diffbot.com · ▲606

    Never write another web scraper

More ai this month

the category →
  • Spira Maxima▲578

    Video model that turns script into viral social media videos

    AI · 18d ago · spira.ai

  • Rill Browser▲540

    The browser where Claude Code and Codex work beside you

    AI · 18d ago · rill.love

  • Germany's new sovereign AI model Kolibri▲424

    Kolibri is Aleph Alpha

    AI · 20d ago · tej.as

  • Giving Opus 5.5 a simulated paint canvas▲392

    Paintings by AI models: programs run through a simulation of oil paint.

    AI · 21d ago · stillwet.art

  • PixVerse R2▲381

    A real-time world model you can explore and change

    AI · 30d ago · world.pixverse.video

  • Beam.win▲378

    Run AI agents on one Mac and use its windows on another

    AI · 4d ago · beam.win

Source: Show HN post ↗

Launched alongside, September 2018

the whole month →