In terms of NodeJS vs Python, specifically for web scraping, would you choose NodeJS? If so, why?
I'm more familiar with NodeJS but I'm working with a team that is leaning towards using Python for web scraping, so that's why I'm asking. They said spinning up multiple processes in Python is easier so at scale it will work better.
I know you can use the Cluster module to have child processes in NodeJS, but in my experience it's a bit of a pain to use, although it's not always required to use anyway, at least when only using NodeJS as a web server (as long as you have multiple NodeJS instances, in case one goes down). Web scraping is a bit different though.
I'm more familiar with NodeJS but I'm working with a team that is leaning towards using Python for web scraping, so that's why I'm asking. They said spinning up multiple processes in Python is easier so at scale it will work better.
I know you can use the Cluster module to have child processes in NodeJS, but in my experience it's a bit of a pain to use, although it's not always required to use anyway, at least when only using NodeJS as a web server (as long as you have multiple NodeJS instances, in case one goes down). Web scraping is a bit different though.
Curious if you have any thoughts on this.