tomsonoda

Thank you! That's a clean design.

The concept of per-user key wrapping is interesting.

I learned this the hard way... just because a wrapped key exists on the server doesn't guarantee the recipient can actually unwrap it—for instance, if the recipient's identity key has changed due to a re-installation. Implementing a mechanism to verify successful decryption on the client side before discarding the old key helped prevent situations where users would lose access to their own data.

There are some specific point to watch out for when implementing an on-device photo processing pipeline—points I encountered firsthand and that you might want to check.

EXIF Orientation: If you decode an image without applying the orientation information, the resulting embeddings will be calculated based on the rotated image. CLIP is surprisingly sensitive to this, and OCR generally fails to work on text rotated by 90 degrees.

It is certainly a benefit that the server only ever sees encrypted data. I have also worked on E2EE for multi-device shared spaces, and the trickiest bugs weren't in the encryption itself, but in edge cases related to key distribution. I have two questions regarding this:

1. When a new client joins but fails to retrieve the workspace key (due to temporary network issues or a 404 error from the server, for example), does it ever generate a new key locally? We encountered exactly this scenario: a device "helpfully" created its own key and encrypted data with it, rendering the message unreadable to other members (without any error notification). We ended up changing the server behavior so that instead of returning a "key missing" response, it signals that the key exists and the client should wait for it.

2. How do you handle key revocation? If you invite someone and later remove them, do you rotate (update) the workspace key and re-wrap it for the remaining members, or does the old key still allow access to past content?

Interesting. Is it also possible to run on Apple Silicon as well as RTX?