maxtaco

I'm not sure exactly what system you're describing, but I hope it doesn't involve hand-marshaling and unmarshaling of data structures. Your requirement (b) seems at odds with forward-compatibility. Lack of forward-compatibility complicates upgrades, especially in federated systems when you cannot expect all nodes to upgrade at once.

I might be biased, but it's been possible to write a sufficiently complex system (https://github.com/foks-proj/go-foks) without feeling "tortured." It's actually quite the opposite, the cleanest system I've programmed in for these use cases, and I've tried many over the last 25 years. Never am I guessing how to verify an object, I'm not sure how that follows.

I also think it's worth noting that the same mechanism works for MAC'ing, Encryption and prefixed hashing. Just today I came across this IDL code:

  struct ChunkNoncePayload @0xadba174b7e8dcc08 {
    id @0 : lib.FileID;
    offset @1 : lib.Offset;
    final @2 : Bool;
  }

And the following Go code for making a nonce for encrypting a chunk of a large file:

  pld := lcl.ChunkNoncePayload{
    Id:     fid,
    Offset: offset,
    Final:  isFinal,
  }
  hsh, err := core.PrefixedHash(&pld)
  if err != nil {
    return nil, err
  }
  var ret [24]byte
  copy(ret[:], (\*hsh)[0:24])
  return &ret, nil

As in signing, it's nice to know this nonce won't conflict with some other nonce for a different data type, and the domain separation in the IDL guarantees that.

Comment · 7 months ago

Bingo!

Comment · 7 months ago

It seems like in that PR, the fact that the OID wasn't checked is part of the problem. I think a better system wouldn't compile or would always fail to verify if the OID (domain separator) is wrong, and I think you'd get that behavior in the posted system.

Comment · 7 months ago

Hi, post author here. Agree that the idea isn't tricky, but it seems like many systems still get it wrong, and there wasn't an available system that had all the necessary features. I've tried many of them over the years -- XDR, JSON, Msgpack, Protobufs. When I sat down to write FOKS using protobufs, I found myself writing down "Context Strings" in a separate text file. There was no place for them to go in the IDL. I had worked on other systems where the same strategy was employed. I got to thinking, whenever you need to write down important program details in something that isn't compiled into the program (in this case, the list of "context strings"), you are inviting potentially serious bugs due to the code and documentation drifting apart, and it means the libraries or tools are inadequate.

I think this system is nice because it gives you compile-time guarantees that you can't sign without a domain separator, and you can't reuse a domain separator by accident. Also, I like the idea of generating these things randomly, since it's faster and scales better than any other alternative I could think of. And it even scales into some world where lots of different projects are using this system and sharing the same private keys (not a very likely world, I grant you).

Comment · 7 months ago

It should be possible to change the name of the type, and this happens often in practice. But type renames shouldn't break preexisting signatures. In this scheme you are free change the type name, and preexisting signatures still verify with new code -- of course as long as you never change the domain separator, which you never should do. Also you'd need to worry about two different projects reusing the same type name. Lastly, the transmission size in this scheme remains unaffected since the domain separators do not appear in the serialized data. Rather, both sides agree on it via the protocol specification.

Comment · 7 months ago

I would say two problems with the asn.1 approach are: (1) it seems like too much cognitive overload for the OIDs to have semantic meaning, and it invites accidental reuse; I think it matters way more that the OIDs are unique, which randomness gets you without much effort; and (2) the OIDs aren't always serialized first, they are allowed to be inside the message, and there are failures that have resulted (https://nvd.nist.gov/vuln/detail/cve-2022-24771, https://nvd.nist.gov/vuln/detail/CVE-2025-12816)

(edit on where the OIDs can be, and added another CVE)

Comment · 10 months ago

Also on the front page of HN right now is a job posting for Optery (YC W22). Seems like they are growing really fast.

Comment · 10 months ago

An intended use case for FOKS (https://foks.pub) is to allow long-lived durable shared secrets between users and teams with key rotation when needed.