nowfound

Dev tools · June 18, 2019

Refresh Crawler - website refresh tool

Crawls and indexes a site then refresh all its pages

Refresh Crawler - website refresh tool launched on June 18, 2019 with 3 votes.

Visit site ↗Alternatives

What it does

Crawl and Refresh thousands of web pages in one click. Easily & automatically reload and refresh web sites. You can choose which level of depth to crawl to, how often to refresh, when to stop and you can even configure it to crawl and refresh linked sites.

Does a similar job

all alternatives →
  • Crawlee2022 · ▲229

    Build reliable web scrapers and robots, fast!

  • CW
    Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

    Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

  • Crawlr By Crawlr Labs2024 · ▲8

    Effortlessly scrape web data with Crawlr.

  • SC
    SiteOne Crawler – in-depth website analyzer and exporter (open-source)2024 · github.com · ▲32

    Please try out this website analyzer and provide feedback on how to improve the output report so that you love the tool. In the first few sentences of the README you will also find a link to the GUI application if you don't want the CLI. To give you an idea of the report, I am sending a sample HTML report for Apple.com - https://crawler.siteone.io/html/2024-08-19/forever/v-bnb7tu5...

  • P3
    Python 3.5 Async Web Crawler Example2016 · github.com · ▲28
  • Crawlio2025 · ▲13

    Scrape, crawl, and automate the modern web

More dev tools this month

the category →
  • Meridian▲530

    Don't let your work go unnoticed. Get promoted!

    Dev tools · 25d ago · meridiona.com

  • x1▲516

    Lovable for iPhone apps go from idea to App Store

    Dev tools · 16d ago · x1.new

  • Open-Source eInk Bike Computer▲421

    OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.

    Dev tools · 7d ago · opentrailpaper.com

  • Nuphos▲380

    The AI-Native DevOps Workspace.

    Dev tools · 29d ago · nuphos.ai

  • Toplify▲365

    Track your App Store ranking worldwide

    Dev tools · 20d ago · toplify.app

  • Vendo ▲362

    Let your users build their own features inside your product

    Dev tools · 25d ago · vendo.run

Source: Product Hunt launch ↗

Launched alongside, June 2019

the whole month →