shadaj
Joined 10 years ago
Karma
159
Submissions
30
Member since
Dec 13
Recent submissions
Comment · 1 year ago
You might enjoy my first ever blog post from ~10 years ago, when I first learned about distributed systems: https://www.shadaj.me/writing/romeo-juliet-and-reactive-prog...
Comment · 1 year ago
You caught me! That's what my next post is about :)
Comment · 1 year ago
Erlang (is great but) is still much closer to the static-location (Actors) paradigm than what I’m aspiring for. For example, if you have stateful calculations, they are typically implemented as isolated (static-location) loops that aren’t textually co-located with the message senders.
Comment · 1 year ago
Stay tuned for the next blog post for one potential answer :) My PhD has been focused on this gap!
Comment · 1 year ago
You caught us in our docs-writing week :) In the meantime, the Rustdoc for streams are fairly complete: https://hydro.run/rustdoc/hydro_lang/stream/struct.Stream
Comment · 1 year ago
Flo lead-author here! This is spot on :) Flo aims to be a bit less opinionated than Timely in how the runtime should behave, so in particular we don't support the type of "time-traveling" computation that Timely needs when you have iterative computations on datasets with retractions.
This is also one of the core differences of Timely compared to DBSP, which uses a flat representation (z-sets) to store retractions rather than using versioned elements. This allows retractions to be propagated as just negative item counts which fits into the Flo model (and therefore Hydro).
Comment · 1 year ago
Currently, Hydro is focused on networked applications, where most parallelism is across machines rather than within them. So there is some extra overhead if you want single-machine parallelism. It's something we definitely want to address in the future, via shared memory as you mentioned.
At POPL 2025 (last week!), an undergraduate working on Hydro presented a compiler that automatically compiles blocks of async-await code into Hydro dataflow. You can check out that (WIP, undocumented) compiler here: https://github.com/hydro-project/HydraulicLift
Comment · 1 year ago
These code examples aren't fully documented yet (which is why we've not linked them in the documentation), but you can take a look at a (more-real) implementation of Paxos here: https://github.com/hydro-project/hydro/blob/main/hydro_test/.... We're also working on building more complex applications like a key-value store.
Comment · 1 year ago
Hi, I'm one of the PhD students leading the work on Hydro!
DFIR is more of a middle-layer DSL that allows us (the high-level language developers) to re-structure your Rust code to make it more amenable to low-level optimizations like vectorization. Because DFIR operators (like map, filter, etc.) take in Rust closures, we can pass those through all the way from the high-level language to the final Rust binaries. So as a user, you never interact with DFIR.
https://shadaj.me