This is very neat! Last year I had to essentially do this on GCP and relied on a very similar implementation. Everyone was surprised to see JS being used for data processing but it worked wonderfully.
One thing I want to ask is the retries, how do you handle that currently? I ran into multiple cases where functions would fail for transient reasons.
One thing I want to ask is the retries, how do you handle that currently? I ran into multiple cases where functions would fail for transient reasons.