You can't always add a few more servers. For example, let's say you have a per-request processing budget of 100ms.
The efficiency of the generated code for your language of choice effectively limits the amount of (non-concurrent, single threaded) code you can run while processing said request.
It's exactly this limit that eventually leads engineering teams at your Dropboxes, Facebooks, Twitters, Tumblrs, etc. to start investigating alternatives or alternative implementations of their favorite dynamic languages (PHP/Python/Ruby -> HHVM/Pyston/?).
The efficiency of the generated code for your language of choice effectively limits the amount of (non-concurrent, single threaded) code you can run while processing said request.
It's exactly this limit that eventually leads engineering teams at your Dropboxes, Facebooks, Twitters, Tumblrs, etc. to start investigating alternatives or alternative implementations of their favorite dynamic languages (PHP/Python/Ruby -> HHVM/Pyston/?).