nowfound

Dev tools · January 20, 2021

Plugin Based, Batteries Included, Web Scraper

Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom. - get-set-fetch/scraper

Plugin Based, Batteries Included, Web Scraper launched on January 20, 2021 with 5 votes, #1,387 of 3,157 launches that month and more than 50% of that year's launches. The site was still online in September 2026, 5 years on.

Visit github.com ↗Alternatives

Does a similar job

all alternatives →
  • Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

    Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

  • Getting started with Puppeteer and Chrome Headless for Web Scraping2017 · github.com · ▲142

    Getting started with Puppeteer and Chrome Headless for Web Scraping - emadehsan/thal

  • scrape-it2016 · github.com · ▲186

    A Node.js scraper for humans.

  • Scraperjs – A versatile web scraper2014 · github.com · ▲192

    A complete and versatile web scraper. Contribute to ruipgil/scraperjs development by creating an account on GitHub.

  • Web scraping that just works with OpenFaaS with Puppeteer2020 · openfaas.com · ▲145

    Learn how to scrape webpages using Puppeteer and Serverless Functions built with OpenFaaS.

  • Apify SDK – A scalable web crawling and scraping library for JavaScript2018 · github.com · ▲78

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headf

More dev tools this month

the category →
  • /dev/null▲490

    a social network for devs

    Dev tools · 3d ago · dev-null.sh

  • Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

    open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

    Dev tools · 28d ago · github.com

  • Reladraw – A diagram language where you decide where to place things▲411

    Contribute to reladraw/reladraw development by creating an account on GitHub.

    Dev tools · 26d ago · github.com

  • OpenBot▲362

    Grok Bot alternative: free, local, open-source, multiplayer

    Dev tools · 23d ago · openbot.run

  • Talanoa▲346

    Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

    Dev tools · 21d ago · talanoa.email

  • CoreSpeed▲332

    One MCP for everything your agents need: apps, memory, tools

    Dev tools · 29d ago · corespeed.io

Source: Show HN post ↗

Launched alongside, January 2021

the whole month →