Comment

Comment by WJW

A conduit pulls information from earlier conduits in the same pipeline, only requesting the next item when it needs to do more work in order to process a request for a value from a conduit later in the pipeline. This means that inside the conduit itself there is perfect backpressure: you cannot overload any buffers between conduits because there aren't any.

Of course, depending on the rest of the system you could still have a bottleneck somewhere. For example: if you pull jobs from a queue in (say) Redis but don't have enough capacity to process jobs faster than they are enqueued, the queue will eventually fill up.

parent WJW 3 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton