ub-volta-toss
Joined 2 years ago
Joined 2 years ago
The Temporal team has acknowledged that Cassandra-backed Temporal hits scaling limits pretty fast.
The limitations aren't a clean "X actions/sec", they're sneakier. Because you can run X/sec for days and then the memory on the history service will spike, or any tiny slowdown in the DB will cause looping degradation. There are nasty feedback loops hidden in Temporal that turn small problems into very very large problems.
I think the core problem with Temporal is the way its sharded. This affects history service and its caches. If anything tells them to reload or restart, or that any of the nodes are unreachable, you get a retry storm on the DB.
In addition to these issues, Temporal can create feedback loops within itself. I've seen cases where it would not return to health, even with 0 workers requesting work for 10s of minutes.
We could have kept using and scaling Temporal, but it required 10-30x the resources of building something else. And it was scary to administer. You really need an entire team. You can't have somebody who isn't a dedicated engineer take on-call for it.
After a year of high-scale Temporal work, I found it was only good for low-scale work.
The onboarding and learning curve were insanely difficult and complex. Ultimately it doesn't scale as well as you think. The temporal team invented their own database to get around this limitation.
- Nobody who works in a lab would call screw cap tubes 'these little tubes'
- Shows no action from machine
- How is machine supposed to open 'these little tubes'? It can't unscrew them.
- If the operator starts the machine with open screw-cap tubes (usually full of expensive things) how would the operator (or robot) close the tube for the rest of the run? Leaving it open is bad, seems like opening the machine should be bad...
I think if this company actually had a working machine, they'd show it working
edit: glassdoor reviews are insaaaane. if people are willing to leave reviews like that at a company this small, its gotta be worse than they're even describing.
so much promise in the bio space absorbing tech, so much peril when tech tries to bio
i think nato built something like this at uBiome, but it never saw use. another yc that scrubbed the glass-door reviews
A friend who struggles with depression found it useful for a while, but its effects didn't continue after a number of doses.