lorendsr

Temporal is designed to handle lower latency use cases than data pipeline systems like Airflow. It also has added a feature recently called Update designed for request-response style interactions that allows for communication with Workflows on the order of tens of ms.

There are certainly use cases for which it's more than is required. Like the most simple would be adding a cron string to a GitHub Action or Vercel function, but in most cases, and certainly Slack's case, you want more reliability, scalability, flexibility, and/or observability. And Temporal has extreme levels of those things, including pausing & editing schedules and seeing the current status of all triggered scripts/functions and all the steps a function has taken so far, and guaranteeing the triggered function completes, including ensuring that if the process or container dies, the function continues running on another one. Even if you don't care about all those things, you might care about some of them in the future, and it doesn't hurt to run a system that has capabilities you don't use.

Thanks!

You do need to run it on the same code version. There are different ways deploy code changes. If you use one of our in-built versioning systems, the version is recorded in the workflow history (and you can either keep track of version-sha mapping or use a sha as the version). Otherwise, you can add code that adds the current code version as workflow metadata.

Author here, curious if there are any major TTD debuggers I missed? Also let me know if anything in the post didn't make sense, and I'll edit to clarify!

2PC tends to have limited throughput due to the participants needing to hold a lock between the voting and commit phase, and all the participants need to support the protocol. Sagas work across different services and data stores and can have high throughput.

However, if all of the data you need to update is in a single database that supports atomic commits, I'd go with that over sagas.

At some point, if you can't automatically fix something, you have to stop and report to a human for manual intervention/repair. While a saga doesn't guarantee that you avoid manual repair, it significantly reduces the need for it. If each of these has a 1% chance of non-retryable failure:

Step1

Step2

Step1Undo

then this has a 1% chance of needing manual repair (it's okay if step1 fails, but if step1 succeeds and step2 fails, we need to repair):

do Step1

do Step2

and this has a .01% chance (we only repair if Step2 and Step1Undo fails, 1% * 1%):

do Step1

try {

  do Step2

} catch {

  do Step1Undo

}

If I'm getting your point right, I agree! If you have the workflow / durable execution primitive to depend on (a durable function is guaranteed to complete executing), then there are a lot of pieces of distributed systems stacks that you no longer need to use. Your durable code is automatically backed by the event sourcing, timers, task queues, transfer queues, etc that Temporal internally uses to provide the guarantee, so that you don't need to build them yourself.

There are definitely ease of use benefits to more tailored solutions. If workflow definitions are really simple and don't change much, JSON might be easy. Most things I prefer the DX of writing the logic in code. And it wouldn't be 50 different codebases—it would usually be a single codebase with 50 different functions.

Temporal has different database options: Cassandra, Postgres, MySQL, SQLite.

> source data in CSV files; - Capillaries script (JSON file) that defines the workflow and the transforms; - Python code that performs complex calculations (only if needed).

Temporal is more general purpose: source data anywhere, and you write code to define workflows and transforms instead of JSON, and the code can be in Go/Java/Python/JS/TS/.NET

Temporal also shares the principle of being tolerant to database and processing node failures