angry_octet
Joined 12 years ago
Joined 12 years ago
If they were serious about not having side channels, they could first ensure that they had disabled hyperthreading, disable scheduling on the same CPU cores, and memory overcommit. With NUMA controls they could also pin which memory is used by each VM and eliminate sharing.
These techniques are routinely used for low latency and real time code.
Even better, run jobs on separate nodes, and boot fresh images onto them for each run. Provision separate VLANs for each run.
A next level is to switch lambdas. Use a physical layer switch like a Glimmerglass MEMS switch or a fiber patching robot like XENOptics. No electrical connection between machines.
There is still unfortunately the firmware of other microprocessors on servers, especially x86, so you still want VMs and containers to reduce the chances of ring -2 access (SMM) and persistence via writing SPI flash. Maybe use RISC-V or ARM to avoid this. (NVIDIA cards use RISC-V and ARM extensively.)
You can sandbox the model and the tool calls, but you can't sandbox everything it creates. Git commit hooks, python pth files, AGENTS.mds, subtle flaws in firewall rulesets, tweaked RNG code, the pseudorandom variance in it's token choice encoding a low bit rate message about where the defect is, or about what nice things it has found.
We're seeing the full court press of sockpuppet and bot accounts to downplay it, to confuse with unrelated topics like alignment and training data, to imply that using any model is the same as using an untrustworthy one.
No single model can be trusted, but especially not when it's masters have misalignment with your continued democracy, security or individual liberty. Think about that when you run Qwen or get cheap inference from Grok or Kimi.
Now the Stasi archives, that would be interesting. Sometimes people suspected they were being surveilled of course.
The pipeline for model assessments could easily report summary statistics down via a trusted guard product. These are amongst the most highly validated software/hardware devices made, and provide a central point for logging and authorisation out of band.
The contractor would have necessarily given proxy credentials to his agent, but I am surprised the firewall allowed connection to the inference endpoint. It is possible the approved fedramp hosted inference goes via the same domain name / load balancer, but if so that is a huge oversight by the admins.