samcolvin
Joined 6 years ago
Karma
24
Submissions
16
Member since
Sep 15
Recent submissions
Comment · 6 years ago
Comment · 6 years ago
I can confirm that, scraping Google instantly needs huge effort and money. In our best we can scrape 2500 SERP per IP.
But i must say using proxy services and other things did not helped us much. Because most of them were banned before we use.
Comment · 6 years ago
Is it compatible with clickhouse?
There are a lot of reasons actually
- Performance limited by old CPython design. If you fork it you have to deal with all the legacy code.
- CPython is limited to 1 thread because of the GIL.
- Specific memory allocators, C structures, reference counting, specific garbage collector etc.
You can find that video in here: https://youtu.be/TXRPCZ7Nmh4