Or use a language where fully utilizing all CPU cores is transparent, like Elixir? There's zero complexity, you basically add 4-5 lines of code and that's it. Honestly, not exaggerating.
I've done several very amateur scrapers in the last several years, I am never going back to languages with a global interpreter lock, ever.
I've done several very amateur scrapers in the last several years, I am never going back to languages with a global interpreter lock, ever.