This looks really neat, I love the idea of a single api for both traditional and headless scraping.
From my experience headless scraping is in the order of 10-100x slower and significantly more resource intensive, even if you carefully block requests for images/ads/etc.
You should always start with traditional scraping, try as hard as you can to stick with it, and only move to headless if absolutely necessary. Sometimes, even if it will take 10x more “requests” to scrape traditionally, it’s still faster than headless.
From my experience headless scraping is in the order of 10-100x slower and significantly more resource intensive, even if you carefully block requests for images/ads/etc.
You should always start with traditional scraping, try as hard as you can to stick with it, and only move to headless if absolutely necessary. Sometimes, even if it will take 10x more “requests” to scrape traditionally, it’s still faster than headless.