My team helps run and deploy a python service that is entirely CPU bound. It accepts an input, performs some computation, and returns a result without any sort of I/O outside of the initiating HTTP request. In the past week it's averaged around 144 req/s with a p95 latency of ~1s.
We average ~80 "instances" to maintain this level of performance. I have very little doubt that, if given the opportunity to rewrite this in rust, we could smash 10x perf improvements. Could we also get more perf out of tuning our python code better? Definitely. Do I think there's 10-20x improvement waiting to be uncovered? No.
Unfortunately (fortunately?) we're at a stage that it makes more sense to throw ludicrous sums of money at it than it does to ground up rewrite.
We average ~80 "instances" to maintain this level of performance. I have very little doubt that, if given the opportunity to rewrite this in rust, we could smash 10x perf improvements. Could we also get more perf out of tuning our python code better? Definitely. Do I think there's 10-20x improvement waiting to be uncovered? No.
Unfortunately (fortunately?) we're at a stage that it makes more sense to throw ludicrous sums of money at it than it does to ground up rewrite.