Same result would be achieved by paying tons of $$ for a high-end machine with dozens of cores, just to scrape one URL per core.
They could have implemented asyncIO within just a few Lambdas - heck even one Lambda could certainly handle hundreds, if not thousands of ASYNC jobs at once, just as a cheap machine, as you pointed.
They treated Lambda functions as threads or async jobs, while they should be looked at as core processors.
Same result would be achieved by paying tons of $$ for a high-end machine with dozens of cores, just to scrape one URL per core.
They could have implemented asyncIO within just a few Lambdas - heck even one Lambda could certainly handle hundreds, if not thousands of ASYNC jobs at once, just as a cheap machine, as you pointed.
They treated Lambda functions as threads or async jobs, while they should be looked at as core processors.