nowfound

Dev tools · January 9, 2021

CH

Cloakbits - Headless web scraping with bypass for anti-bot WAFs

There is a growing number of companies offering anti-bot protection SaaS to protect websites from scraping by automated bots based on Puppeteer/Selenium. Most of them rely on browser properties such as headers, javascript properties (window., navigator.), behavior analysis, to build device/user fingerprints and match it against a database of "whitelisted" fingerprints (typical user behavior/settings/device props etc). For the past few months, together with two other devs I have worked on a customized Puppeteer/Playwright scraping backend. It's essentially a drop-in…

Alternatives

What it does

In the maker’s words, at launch

There is a growing number of companies offering anti-bot protection SaaS to protect websites from scraping by automated bots based on Puppeteer/Selenium. Most of them rely on browser properties such as headers, javascript properties (window., navigator.), behavior analysis, to build device/user fingerprints and match it against a database of "whitelisted" fingerprints (typical user behavior/settings/device props etc). For the past few months, together with two other devs I have worked on a customized Puppeteer/Playwright scraping backend. It's essentially a drop-in replacement for default Chrome/FF binaries. We managed to successfully go through Coinbase, Amazon, Aliexpress login pages in headless mode without getting captcha, or any other verification. We are planning to roll out a beta version. If you are interested in getting beta access leave us details about your use case here: https://a90eq67iroz.typeform.com/to/FAkWnrtv The motivation for our project is that open-source solutions such as puppeteer-extra-stealth cover only a small portion of what popular anti-bot software such as Akamai Bot Manager or Imperva use to detect and ban emulated browsers.

Does the same job

all alternatives →

More dev tools this month

the category →
  • Dograh▲592

    The open source VAPI alternative

    Dev tools · 25d ago · dograh.com

  • Meridian▲530

    Don't let your work go unnoticed. Get promoted!

    Dev tools · 20d ago · meridiona.com

  • x1▲516

    Lovable for iPhone apps go from idea to App Store

    Dev tools · 11d ago · x1.new

  • The GTM Co-Founder▲402

    Open-source GTM skills for technical founders

    Dev tools · 29d ago · gtmcofounder.com

  • Open-Source eInk Bike Computer▲393

    OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.

    Dev tools · 2d ago · opentrailpaper.com

  • Nuphos▲380

    The AI-Native DevOps Workspace.

    Dev tools · 24d ago · nuphos.ai

Launched alongside, January 2021

the whole month →