Show HN: Cloakbits - Headless web scraping with bypass for anti-bot WAFs
There is a growing number of companies offering anti-bot protection SaaS to protect websites from scraping by automated bots based on Puppeteer/Selenium. Most of them rely on browser properties such as headers, javascript properties (window., navigator.), behavior analysis, to build device/user fingerprints and match it against a database of "whitelisted" fingerprints (typical user
behavior/settings/device props etc).
For the past few months, together with two other devs I have worked on a customized Puppeteer/Playwright scraping backend. It's essentially a drop-in replacement for default Chrome/FF binaries. We managed to successfully go through Coinbase, Amazon, Aliexpress login pages in headless mode without getting captcha, or any other verification. We are planning to roll out a beta version. If you are interested in getting beta access leave us details about your use case here: https://a90eq67iroz.typeform.com/to/FAkWnrtv
The motivation for our project is that open-source solutions such as puppeteer-extra-stealth cover only a small portion of what popular anti-bot software such as Akamai Bot Manager or Imperva use to detect and ban emulated browsers.
For the past few months, together with two other devs I have worked on a customized Puppeteer/Playwright scraping backend. It's essentially a drop-in replacement for default Chrome/FF binaries. We managed to successfully go through Coinbase, Amazon, Aliexpress login pages in headless mode without getting captcha, or any other verification. We are planning to roll out a beta version. If you are interested in getting beta access leave us details about your use case here: https://a90eq67iroz.typeform.com/to/FAkWnrtv
The motivation for our project is that open-source solutions such as puppeteer-extra-stealth cover only a small portion of what popular anti-bot software such as Akamai Bot Manager or Imperva use to detect and ban emulated browsers.