ScrapydWeb

What it does

Scrapyd Cluster Management *Group, filter and select any numbers of nodes *Execute command on multinodes with one click Scrapy Log Analysis *Stats collection *Progress visualization *Logs categorization All Scrapyd API Supported *Deploy project, Run Spider, Stop job *List projects/versions/spiders/running_jobs *Delete version/project

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right: - my8100/scrapydweb

• 
  ScrapX2023 · scrapx.io · ▲356

  Stay ahead of your competitors - Our service tracks the websites you select on a daily basis and notifies you of any changes, whether it's a new headline or a new product that's been released.

• 
  SCRAPRMar 2026 · scraprbeta.vercel.app · ▲260

  The data layer for the agentic web

• 
  Crawly2016 · crawly.diffbot.com · ▲606

  Never write another web scraper

• 
  Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

  Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

• 
  Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language2019 · github.com · ▲116

  Distributed web crawler admin platform for spiders management regardless of languages and frameworks. - tikazyq/crawlab

• 
  Scrapy Real Time2015 · blog.scrapinghub.com · ▲37

  Introducing ScrapyRT: An API for Scrapy Spiders - Make the most of your Scrapy spiders with ScrapyRT. Explore its functionalities as an API for seamless integration.

• 
  /dev/null▲490

  a social network for devs

  Dev tools · 4d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 29d ago · github.com

• 
  Run full iOS 16.5 firmware on Android Devices▲419

  Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.

  Dev tools · 17h ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 24d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 22d ago · talanoa.email

• 
  Funkify▲1,004

  Experience the web through the eyes of the differently abled

  Life & fun · 2018

• 
  Ubersuggest 2.0▲868

  Neil Patel's free keyword research tool

  Growth · 2018 · neilpatel.com

• 
  Cruip▲696

  Free and premium landing page templates for startups

  Growth · 2018 · cruip.com

• 
  GitHub Actions▲626

  Automate your workflow from idea to production

  Dev tools · 2018 · github.com

• 
  Websites in 2018▲595

  Example of bloated website in 2018

  Life & fun · 2018 · 2018.bloomca.me

• 
  Notion Pages▲579

  Discover new, productive Notion templates

  Work · 2018 · notionpages.com