FabianCarbonara
Joined 5 years ago
Joined 5 years ago
The WebRTC hole actually goes away down there. RTCPeerConnection is Window-only, so it's just absent in a worker. But you're right not to trust CSP: I'm seeing worse. Under default-src/connect-src 'none' in a worker, Firefox still lets EventSource make the request (fetch/XHR/WebSocket/importScripts all block fine).
What I'm experimenting with right now is SES inside the worker: lockdown() + a Compartment. Still figuring out whether my framework stack survives lockdown().
For UI I'm using Shopify's remote-dom to mirror the UI to the trusted parent page.
About the sprites: loading them from Python at runtime actually works. Stage's Bank objects are plain Python bytes (2048 bytes = 16 tiles at 16x16, 4bpp) plus a palette; on first use, _snesstage.bank() converts them to SNES bitplane format and DMAs them into one of 8 VRAM slots.
On extension modules in general: MicroPython has compiled-in C "user modules" (that's what _snesfb/_snesstage are) and dynamically loadable native .mpy modules. But this needs per-architecture relocation support, and the 65816 isn't among the supported targets.
My approach is the opposite bet: full code execution instead of tool calls. The agent can build any React UI from scratch with the full power of code — including client-server data flow, callbacks, streaming data.
MDX is a compile-time format for static content. This is a runtime protocol where the LLM writes code that executes as it streams, and the UIs it creates stay connected to the server.
Combined with a slot mechanism, complex UIs build up progressively — a skeleton appears first, then each section fills in as the LLM generates it.
I wrote a deeper dive on how the streaming execution works technically: https://fabian-kuebler.com/posts/streaming-ts-execution/
Right now this uses React for Web but could also see it in the terminal via Ink.
And I love the "freeze" idea — maybe then you could even share the mini app.
You're right that the level of expressiveness is the key design decision. There's a real spectrum:
- pre-registered blocks (safe, predictable)
- code execution with a component library (middle ground)
- full arbitrary code (maximum flexibility).
My approach can slide along that spectrum: you could constrain the agent to only use a specific set of pre-imported components rather than writing arbitrary JSX. The mount() primitive and data flow patterns still work the same way, you just limit what the LLM is allowed to render.
Would love to hear what you learn if you explore it!