Love this space. I’ve build a few versions of this in go, glad to see some go native frameworks.
Would also love to see some examples.
Concerns I’ve had to deal with in the past (if these are solved, examples would go a long way):
- max queue size between elements is important. Don’t want the database reader role to outrun the workers, but also never want them desaturated
- some dynamic limits for local to a role (limit DB readers to limit connection pool) and global (how much cpu and memory share does a job need before starting)
Would also love to see some examples.
Concerns I’ve had to deal with in the past (if these are solved, examples would go a long way):
- max queue size between elements is important. Don’t want the database reader role to outrun the workers, but also never want them desaturated
- some dynamic limits for local to a role (limit DB readers to limit connection pool) and global (how much cpu and memory share does a job need before starting)