wh33zle

Correct! I played around with this in the past. It helps less than you'd think. As soon as you have a generic resume parameter, you can no longer drive the generator with a generic runtime.

If I am now writing my own runtime / eventloop to advance my state machine anyway, what exactly am I getting from doing that through an abstraction like the Generator trait?

On top of that, the resume and yield abstraction can get convoluted because a lot of times, you only want to do one or the other, meaning either case needs to support a no-op because a single `yield` always does both.

Lastly, with all input going through a single type, your state machine then needs to handle every possible input at every yield point. That is a lot of `match` statements and branching.

I am sure there are better ways to do this and having generators to express custom iterators would be amazing. But for generic state machines, I don't see much of a benefit.

There is actually more thought in it than that.

The quoted section is meant to explain in layman's terms that we are doing hardware offloading where possible. That means vectorized instructions for encryption and checksum-offloading and packet segmentation in the kernel / hardware driver where supported.

Being more efficient per packet means that we using less power to move the same amount of data, hence less power draw on mobile platforms.

Having said that, I agree that it could be better while still not be filled with technical jargon.

In principle, you are right. Async is not I/O. And yes, `async` functions compile to a deterministic state machine.

The problem is, the moment you use `async`, your state machine reduces down to one API: `poll`. What if I want to change some state in-between two state transitions (i.e. between two `await` points)? Now I have to wire-in a channel to supply that data at the right point or access some shared data via a Mutex.

Rust's `async` implementation is great if you know everything upfront before you kick off your state machine. The moment you need to do many things concurrently, it can become a real mess.

Firezone's sans-IO core handles many things concurrently: inbound IP, inbound UDP, WebSocket control messages, TURN allocations, timers, user commands, etc. Having essentially one big struct that I can just call `&mut self` functions on for any one of these inputs makes this very easy to understand, compose and test.

A few things like TURN request-response handshakes might be easier as an `async` function but in reality, it is not a big deal and the composability you get from it outweighs it in my opinion.

You can't use the Linux kernel implementation as soon as you want to be cross platform and the only official userspace implementation is in Go which is awkward to FFI into from Rust.

I agree with your point. Use what is out there. We did that by using Cloudflare's boringtun for a long time. But it has limitations that eventually we could no longer work around if we wanted good compile-time safety and performance.

I can recommend Fastmail. Reasonably priced, modern tech & features, good service. Never had any issues.

Coverage is just the feedback loop. If you don't have a good generator, feedback that certain areas are uncovered doesn't really help. But without coverage, the generator doesn't really explore deeper states.

Under the hood, the fuzzer actually works with _features_, not source code coverage and there are ways in explicitly giving the fuzzer additional feedback. Some early data suggests that this is very effevtive.

How low do you need the footprint? At firezone.dev, we also build a ZT product and our headless-client and Gateway are in Rust and use around 15-30 MB of RAM. Could likely be tuned further down if you need it :)

One of the coolest things I've learned about recently is `.git/info/exclude`. It allows you to ignore files in the local repo without modifying the repo's .gitignore

Very useful if you want to add your own .envrc or shell.nix to a repo.

Depends. MASQUE implies QUIC and a lot of coporate networks still block QUIC, forcing a fallback to TCP (for web browsers).

Not sure how they want to address this in case of WARP?

Nevertheless, MASQUE is some pretty exciting development in the network space.

I find myself defining my own traits very rarely these days. Traits enable polymorphic actions on data. Most of the time, you simply don't need that.

An enum often suffices to express the different combinations.

Of course, from a pattern perspective, that is "not extensible" but it often ends up being easier to maintain and in Rust, often compiles down to more efficient code.

WASM shouldn't care about which language the host is written in vs what the apps are written in.

Writing the host in C seems like a bad choice though. Isn't that exactly the kind of software you want to ensure is memory-safe?

> An async method, basically does the same, but it means your logic doesn't have to handle the polling, orchestration, selection, ordering, etc.

In order to take advantage of that, I'd need to create one `async` method for each handshake with the TURN server, right? Like every `allocate`, every `refresh`, every `bind_channel` would need to be its own async function that takes in a `Sink` and `Stream` in order to get the benefit of having the Rust compiler generate the suspense-point for me upon waiting on the response. Who is driving these futures? Would you expect these to be be spawned into a runtime or would you use structured concurrency a la `FuturesUnordered`?

A single `Allocation` sends multiple of these requests and in the case of `bind_channel` concurrently. How would I initialise them? As far as I understand, I would need to be able to "split" an existing `Stream` + `Sink` pair by inserting another multiplexer in order to get a `Stream` that will only return responses for the messages that have been sent via its `Sink`, right?

I just don't see the appeal of all this infrastructure code to "gain" the code-gen that the Rust compiler can do for these request-response protocols. It took my a while to wrap my head around how `Node` should work but the state isn't that difficult to manage because it is all just request-response protocols. If the protocols would have more steps (i.e. more `.await` points in an async style), then it might get cumbersome and we'd need to think about an alternative.

> > It is doable but creates the kind of spaghetti code of channels where concerns are implemented in many different parts of the code. It also makes these things harder to test because I now need to wire up this orchestrating task with the TURN clients themselves to ensure this response mapping works correctly.

> I read through the node code and it feels a bit spaghetti to me as an outsider, because of the sans-io abstractions, not inspite of it. > That comes from nature of the driving code being external to the code implementing the parts.

It is interesting you say that. I've found that I never need to think about the driving code because it is infrastructure that is handled on a completely different layer. Instead, I can focus on the logic I want to implement.

The aspect I really like about the structure is that the call-stack in the source code corresponds to the data flow at runtime. That means I can mentally trace a packet via "jump to definition" in my editor. To me, that is the opposite of spaghetti code. Channels and tasks obfuscate the data flow and you end up jumping between definition site of a channel, tracing where it was initialised, followed by navigating to where the other end is handled etc.

Thinking about your example again made me think of another requirement: Multiplexing.

STUN ties requests and responses together with a transaction ID that is encoded as an attribute in the message.

Modelling this using `Sink`s and `Stream`s is quite difficult. Sending out the request via a `Sink` is easy but how do you correctly dispatch the incoming response again?

For example, let's say you are running 3 TURN clients concurrently in 3 async tasks that use the referenced `Sink` + `Stream` abstractions. How do you go from reading from a single UDP socket to 3 streams that each contain the correct responses based on the transaction ID that was sent out? To read the transaction ID, you'd have to parse the message. We've established that we can do that partially outside and only send the `Result` in. But in addition to the 3 tasks, we'd need an orchestrating task that keeps a temporary state of sent transaction IDs (received via one of the `Sink`s) and then picks the correct `Stream` for the response.

It is doable but creates the kind of spaghetti code of channels where concerns are implemented in many different parts of the code. It also makes these things harder to test because I now need to wire up this orchestrating task with the TURN clients themselves to ensure this response mapping works correctly.