demurgos
Joined 10 years ago
Joined 10 years ago
I'm working at a company with a very large Rust codebase with thousands of endpoints. We use multiple repos and each service publishes its API as crates. A few years ago those were manually written and teams used exhaustive enums frequently in their API. It did not prevent runtime issues as older clients could receive responses with new variants from newer servers. It also made versioning painful because semver would dictate a major bump, but Rust's nominal typing means that types from different major versions get different identities, making it very inconvenient to integrate with these types (think shared helper functions, extension traits, ...). This caused people to release new variants in feature bumps and wreck havoc in the dependency tree by breaking compilation of intermediate libs.
Since then we switched to code generation from an IDL for the API model. I was leading this project and decided to produce non_exhaustive enums for the Rust code: it was 2 years ago and the situation drastically improved.
We have a CI step running lints with `non_exhaustive_omitted_patterns` mentionned in an other comment. That's the right approach for such a public API. The big difference is that is that warnings are local to the crate you're working on, so they're more actionable than errors.
This was all in the context of modeling the API of a distributed system. I still tend to use non_exhaustive more often now even for regular helper libraries. It always requires some thought as you need to plan for the forward compat of your code.
At it's core, it's a game about exploration to understand what's happening. I recommend looking around and being curious to enjoy it, and avoid rushing. It's my favorite game.
To give you an estimate, I completed the base game with all secrets in about 20-30h. There's also a DLC called "Echoes of Eyes" adding a new area to explore. In total, I spent 45h to fully complete the game.
In practice, there's no real issue with using the first non-zero component to define the group of API-compatible releases and most package managers agree on the semantics.
There's not much, but I keep a few articles and games there.
The core idea behind the token-based defense is to prove that the origin server had access to the value in the first place such that it could have sent it if the browser didn't add it automatically.
I tend to agree that the inclusion of cookies in cross-site requests is the wrong default. Using same-site fixes the problem at the root.
The general recommendation I saw is to have two cookies. One without same-site for read operations, this allows to gracefully handle users navigating to your site. And a second same-site cookie for state-changing operations.
> You conveniently left out where the Foo string is computed from something that actually need computation.
I left it out because the comment I was replying to was pointing that some logs don't have params.
For the approach using a `Template` class, the expectation would be that the doc would call out why this class exists in the first place as to enable lazy computation. Doing string concatenation inside a template constructor should raise a few eyebrows when writing or reviewing code.
I wrote `logger.log(new Template("foo"))` in my previous comment for brevity as it's merely an internet comment and not a real framework. In real code I would not even use stringy logs but structured data attached to a unique code. But since this thread discusses performance of stringy logs, I would expect log templates to be defined as statics/constants that don't contain any runtime value. You could also integrate them with metadata such as log levels, schemas, translations, codes, etc.
Regarding args themselves, you're right that they can also be expensive to compute in the first place. You may then design the args to be passed by a callback which would allow to defer the param computation.
A possible example would be:
const OPERATION_TIMEOUT = new Template("the operation $operationId timed-out after $duration seconds", {level: "error", code: "E_TIMEOUT"});
// ...
function handler(...) {
// ..
logger.emit(OPERATION_TIMEOUT, () => ({operationId: "foo", duration: someExpensiveOperationToRetrieveTheDuration()}))
}
This is still not perfect as you may need to compute some data before the log "just in case" you need it for the log. For example you may want to record the current time, do the operation. If the operation times out, you use the time recorded before the op to compute for how long it ran. If you did not time out and don't log, then getting the current system time is "wasted".All I'm saying is that `logger.log(str)` is not the only possible API; and that splitting the definition of the log from the actual "emit" is a good pattern.
There are many ideas if you look at SQL libs. In my example I used a different type but there other solutions. Be creative.
logger.log(new Template("foo"))`
logger.log("foo", [])
logger.prepare("foo").log()A logging framework may have the equivalent of prepared statements. You may also nudge usage where the raw string API is `log.traceRaw(String rawMessage)` while the parametrized one has the nicer naming `log.trace(Template t, param1, param2)`.
The only reliable solution would be an allowlist of safe elements and attributes, but it would quickly cause compat issues unless you spend time curating the rules. I did not find an existing lib doing it at the time, and it was too much effort to maintain it ourselves.
The solution I ended up implementing was having a sandboxed Chromium instance and communicating with it through the dev tools to load the SVG and rasterize it. This allowed uploading SVG files, but it was then served as rasterized PNGs to other users.
I am not familiar enough with gcc to know how it impacts out-of-tree free projects or internal development.
The decision was taken a long time ago, it may be worth revisiting it.
[ my public key: https://keybase.io/demurgos; my proof: https://keybase.io/demurgos/sigs/Gc3Y6euEvQ6Yjg3y8KInIRHWRcTgZIqyyp-tC8jQ4nU ]