nowfound

Work · August 23, 2022

CW

Crawlee – Web scraping and browser automation library for Node.js

Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

Visit crawlee.dev ↗Alternativestop 5% of August 2022

What it does

In the maker’s words, at launch

Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using smart sessions, proxies, and browser fingerprints - Simple management and persistence of queues of URLs to crawl - Written completely in TypeScript for type safety and code autocompletion - Comprehensive documentation, code examples, and tutorials - Actively maintained and developed by Apify—we use it ourselves! - Lively community on Discord To get started, visit https://crawlee.dev or run the following command: npx crawlee create my-crawler If you have any questions or comments, our team will be happy to answer them here. [1] https://crawlee.dev/ [2] https://www.youtube.com/watch?v=g1Ll9OlFwEQ [3] https://blog.apify.com/announcing-crawlee-the-web-scraping-a...

Does the same job

all alternatives →

More work this month

the category →
  • Clipto MCP▲640

    Let agents source clips from terabytes of your local video

    Work · 18d ago · clipto.com

  • HyNote for Mac▲383

    Free local transcription that is 100% Private

    Work · 17d ago · hynote.ai

  • VoiceOS App Store▲330

    The app store for voice native apps that lives in your notch

    Work · 28d ago · voiceos.com

  • Scrimba Explain▲325

    Ask any question, get a video back instantly

    Work · 24d ago · scrimba.com

  • The New Calendly▲211

    Handle all of the work before, during, and after meetings

    Work · 17d ago · calendly.com

Launched alongside, August 2022

the whole month →
  • Xnapper▲1,517

    Take beautiful screenshots instantly

    Work · 2022 · xnapper.com

  • Gamma▲1,169

    Write like a doc, present like a deck

    Work · 2022 · gamma.app

  • Threado▲923

    The command center for community builders

    Life & fun · 2022

  • Remotebase▲742

    Build remote engineering teams in 24 hours

    Dev tools · 2022 · remotebase.com

  • DhiWise 2.0▲737

    Convert Figma designs into Flutter 3.0 & React code rapidly

    Dev tools · 2022 · dhiwise.com

  • PA
    Pornpen.ai – AI-Generated Porn▲711

    Hey HN, I've been working on https://pornpen.ai, a site for generating adult images. Please only visit the site if you are 18+ and willing to look at NSFW images. This site is an experiment using newer text-to-image models. I explicitly removed the ability to specify custom text to avoid harmful imagery from being generated. New tags will be added once the prompt-engineering algorithm is fine-tuned further. If the servers are overloaded, take a look at the feed and search pages to look through past results. For comments/suggestions/feedback please visit…

    AI · 2022 · pornpen.ai