Comment by lysecret

Congrats! I had something quite similar in mind (also working a lot in python based streaming ETL). I am unfamiliar with Ray. A few questions: 1. Ray seems to be focussed on ML usecases, are you as well or are you a more generic streaming ETL framework. Can you explain the reasoning behind choosing Ray. 2. I see you deeply integrate Infrastructure (like BQ and Pub/Sub) what is your story on evolution of this infra? What happens if i have deployed infra through your code and I want to edit it? How do you deal with Dev/Prod/Qa stage divide. 3. What is your story on deployment of the "glue" code that runs your pipeline? Do you also handle multi stage pipelines?