C10K is another name for evented I/O, and it's from the
90s. By 2003 thread-per-client was already obsolete and known to be very bad.
It's really quite simple: threads encourage the use of implicit program state via function calls, with attendant state expansion (stack allocation), whereas evented I/O encourages explicit program state, which means the programmer can make it as small as possible.
Smaller server program state == more concurrent clients for any given amount of memory. Evented I/O wins on this score.
But it gets better too! Smaller program state == less memory, L1/2/3 cache, and TLB pressure, which means the server can take care of each client in less time than an equivalent thread-per-client server.
So evented I/O also wins in terms of performance.
Can you write high-performance thread-per-client code? Probably, but mostly by allocating small stacks and making program state explicit just as in evented I/O, so then you might as well have done that. Indeed, async/await is a mechanism for getting thread-per-client-like sequential programming with less overhead: "context switching" becomes as cheap as a function call, while thread-per-client's context switches can never be that cheap.
The only real questions are:
- async/await, or hand-coded CPS callback hell?
- for non-essential services, do you start with
thread-per-client because it's simpler?
The answer to the first question is utterly dependent on the language ecosystem you choose. The answer to the second should be context-dependent: if you can use async/await, then always use async/await, and if not, it depends on how good you are at hand-coded CPS callback hell, and how well you can predict the future demand for the service in question.
It's really quite simple: threads encourage the use of implicit program state via function calls, with attendant state expansion (stack allocation), whereas evented I/O encourages explicit program state, which means the programmer can make it as small as possible.
Smaller server program state == more concurrent clients for any given amount of memory. Evented I/O wins on this score.
But it gets better too! Smaller program state == less memory, L1/2/3 cache, and TLB pressure, which means the server can take care of each client in less time than an equivalent thread-per-client server.
So evented I/O also wins in terms of performance.
Can you write high-performance thread-per-client code? Probably, but mostly by allocating small stacks and making program state explicit just as in evented I/O, so then you might as well have done that. Indeed, async/await is a mechanism for getting thread-per-client-like sequential programming with less overhead: "context switching" becomes as cheap as a function call, while thread-per-client's context switches can never be that cheap.
The only real questions are:
The answer to the first question is utterly dependent on the language ecosystem you choose. The answer to the second should be context-dependent: if you can use async/await, then always use async/await, and if not, it depends on how good you are at hand-coded CPS callback hell, and how well you can predict the future demand for the service in question.