qlong-seattle
Joined 3 years ago
Karma
46
Submissions
15
Member since
Dec 31
Recent submissions
Comment · 1 year ago
Share my recent learnings as an expert of iWF , but a beginner of LLM
Comment · 3 years ago
Took at quick look at apache nifi. Here is my conclusion but anyone feel free to correct me if I am wrong:
I think they only overlap for a small part but quite different:
* Nifi is focusing on moving/sharing datafile (meaning the size could be big) across the system * Nifi does have some logic control flow to act like workflow, but it's not too closed to be a "workflow system" because it missing a lot of capabilities that a typical workflows should have (see "What they all have in common " in https://medium.com/@qlong/iwf-vs-other-general-purposed-work...)
* iWF is a general purposed workflow engine, could conceptually do data sharing but it is not good at sharing the big amount of data or data files * iWF is more specialized at controlling/orchestrating the business
Comment · 3 years ago
We don't have any plan to build other SDKs at this moment.
But it's VERY EASY to build an SDK for iWF so I would encourage anyone interested to build one. I will provide all the support.
A SDK of iWF is very lightweight. It only need to invoke the two WorkflowState APIs from hosting a REST/HTTP service, based on the worker API schema. https://github.com/indeedeng/iwf-idl/blob/main/iwf.yaml#L215
There is no much tricky things inside the SDK that you have to do. For client APIs to call iWF server(the rest of the API defined in the Open API schema), just need to build a wrapper on top of the generated code from Swagger (and it can be optional but it will just make it easier to use).
As a data point, it only took me a week to build GoSDK, including all the tests, by just opening the IntelliJ IDE and also Goland IDE to translate the Java code into Golang.
The whole community including myself will really appreciate it.
Comment · 3 years ago
Yeah this wiki page has exactly what you want to see here:
https://github.com/indeedeng/iwf/wiki/How-to-migrate-from-Ca...
Comment · 3 years ago
There are lots of things that other workflow engines cannot do:
Reset workflow : important to recover from bad states for any reasons. You may have bugs caused this or because downstream outages. Reset can let you go back like timing machines.
Search attributes: let you search for anything like having unlimited database index.
See more details in this article: https://medium.com/@qlong/iwf-vs-other-general-purposed-work...
Comment · 3 years ago
You don't have to run elastic.
Cadence/Temporal let you run only with database for majority of the feature/power. ElasticSearch is only for supporting the searching functionality(advanced visibility).
https://docs.temporal.io/visibility
https://cadenceworkflow.io/docs/concepts/search-workflows/
Same as in iWF, you can use a Temporal service without advanced visibility.
E.g. this doc is showing how to run with Temporalite which doesn't have advanced visibility with it.
https://github.com/indeedeng/iwf/blob/main/CONTRIBUTING.md#o...
Again, you still have 90% power and the only thing missing is searching for workflows using "search attributes".
You can still store data in search attributes but they are just not searchable.
And later, you can migrate to a Cadence/Temporal service that has advanced visibility enabled.
Comment · 3 years ago
I believe so. Welcome to try out. There are only a few concepts you need to learn about iWF and you can start with docker-compose to run a iWF service locally for development.
Comment · 3 years ago
iWF is not just persisting states, but also able invoking/executing states, searching for attributes(like unlimited database indexes), durable timer, resetting workflows etc.
Without Temporal/Cadence, it's 100+ harder to implement iWF. Even using SWF/Azure durable function is possible to build iWF, Cadence/Temporal is much more powerful than the other two(e.g. no reset workflow, search attributes)
Comment · 3 years ago
Maybe you can start with looking at the sample for solving the same "subscription" use case:
Temporal:
https://github.com/temporalio/subscription-workflow-project-...
https://github.com/temporalio/subscription-workflow-project-...
iWF:
https://github.com/indeedeng/iwf-java-samples#product-use-ca...
https://github.com/indeedeng/iwf-golang-samples#product-use-...
Note that you need to consider making changes to workflow -- in Cadence/Temporal, you must use versioning like: https://stackoverflow.com/questions/61298415/how-can-i-versi...
And this versioning/determinism constraints is just a small part of the learning curves for complicated project
And I am not the only one pointing this out. Also see the article from the principal of Azure Durable Function team:
https://medium.com/@cgillum/common-pitfalls-with-durable-exe...
Comment · 3 years ago
There are samples from the README:
https://github.com/indeedeng/iwf-java-samples#product-use-ca...
https://github.com/indeedeng/iwf-golang-samples#product-use-...
Comment · 3 years ago
Hello, no you are definitely the target users. Anyone who needs to build long running business/application/process is the target user of iWF.
You are right, there were too many context about Cadence/Temporal in the README. I just made a update to move a lot of them to wiki just for the Cadence/Temporal users.You can check https://medium.com/@qlong/iwf-vs-other-general-purposed-work... for comparison with other workflow engines.
I will move the G docs and slides later.
Thanks for the suggestion!
Principal Software Engineer @indeedeng . Creator of iWF. Evangelist of Cadence/Temporal workflow. WeChat: prclqz