pqdbr
Joined 14 years ago
Joined 14 years ago
Like, seriously?
From his article:
One backend, two modes
Fiber mode isn’t universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby’s fiber scheduler stall the reactor. And that’s fine – you don’t have to pick one.
As Trevor Turk pointed out in the PR discussion, that’s the whole point: separately configured worker pools. Here’s what Chat with Work actually runs in production:
workers: - queues: [ chat ] fibers: 10 processes: 2 polling_interval: 0.1 - queues: [ turbo ] fibers: 10 processes: 1 polling_interval: 0.05 - queues: [ notifications, default, maintenance ] fibers: 5 processes: 1 polling_interval: 0.2 - queues: [ cpu ] threads: 1 processes: 1
The difference is staggering when you compare to threaded mode: it requires 1,320 database connections to run the same benchmark that the fiber mode runs with 60.
https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/
Use a workflow to implement this. You (Fable) are the orchestrator, planner and reviewer. Opus agents are implementers.
That's all there's to it. it will create the dynamic workflow - has a nice interface native to Claude Code - and do all the coordination to deliver what you asked.
I'm curious: what's _your_ defense against this?
- We have 2 sources of data that we must backup to continue existing as a business; our postgres and binary files in S3. Everything else is derivable (elasticsearch, so on).
- For postgres, we use barman. With the help of opus/fable, you can get a streaming replication backup working in no time. We have one into another server in the same datacenter (we use baremetal) and another one in another server in a different datacenter.
- We then have a last resort barman backup with bi-weekly base backups + WAL streaming to S3 (both the base backup and WALs). It sends these backups + wal segments into an specific S3 bucket that has object lock in compliance mode. This is a feature from AWS S3 that even the most privileged account credentials (super admin) can't turn off nor delete the files before the object lock, which is 10 days in our case. Object lock compliance mode can only be extended, never shortened.
- For S3, we store them into another versioned bucket, with lifecycle rules to also expire non current versions (== deleted objects) after 10 days. No point in object lock compliance here because it would only protect objects for the most recent 10 days, and you gain nothing. What we do instead: the app servers only have access to these bucket tru an IAM credential that can't delete old versions (so deleted objects have to expire manually via the lifecycle rule) AND this IAM credentials also can't change the object policy.
IMHO, this protects us enough so that even in the worst case scenario (ransomware) we have 10 days to sort everything out and recover our AWS access.
And yes, we test the S3 barman restoration and it works fine. Data loss is at max 5 minutes due to the archive_timeout=300s on the primary.
For the streaming replications in the two servers I mentioned, it's less <1ms, but those wouldn't protect us much in the case of the ransomware - even tough we use tailscale and one compromised server can't ssh into the other.