polyglotfacto
Joined 6 years ago
Joined 6 years ago
First, the least relevant metric: LOC. Almost doubled from 7k to 12k in a week.
Current WPT pass rate, compared to the default integration of rustnn (pywebnn) is at: https://gist.github.com/gterzian/95371d56468390177982c068480...
This week was the grunt work: grinding through the long list of missing operations, with almost all now implemented.
Also switched to a more expensive coding model (GPT 5.3 Codex), which was an improvement, but there were still a few clear fails—specifically around the model taking "shortcuts" with spec logic.
What surprised me in this experiment with Lean was that the AI-assisted formalization actually caught an error in my "vetted" TLA+ spec that had been sitting there for two years. It turns out my inductive invariant wasn't strong enough—something the TLA+ model checker didn't slap me for, but Lean’s kernel refused to ignore.
This was also a way to bridge a higher-level spec, written in Lean in a kind of TLA-like flavor, and the lower level implementation, also written in Lean but with a code-like flavor, and then proven that the implementation refines the spec.
Using Lean to bridge the gap between high-level temporal logic and low-level IO implementation, all written by AI but driven by human intent, feels like the first time the "coding singularity" might actually lead to better software rather than just more software
Given the policy in place, I'm doing this on a personal branch of Servo. I’m using it as a sandbox to test a specific thesis: AI is not an architect; it is a "fluid syntax engine."
This is also my response to the "autonomous slop" seen in projects like fastrender that claim to build complex software with AI but produce what I call "fine-looking nonsense" (see https://news.ycombinator.com/item?id=46624541#46709191). I wanted to see if a human architect could use an AI agent to handle the syntax and boilerplate of a Web Standard while the human retains total control over the conceptual structure.
Progress & Tech Stack:
The scope: I am using `rustnn`, and this project can be seen as doing for the Web what `pywebnn` does for Python.
The model: I am intentionally using a "cheap" LLM: Raptor mini.
The scale: I’m 7k LOC into the implementation. It currently passes a subset of WPT (Web Platform Tests) conformance tests for a specific operator (and its dependencies): add. Such a conformance test requires implementing the full graph compilation and dispatch workflow. Code is at https://github.com/gterzian/servo/compare/master...gterzian:servo:webnn
The backend: The work involves integrating the `rustnn` library into Servo’s runtime, with compute currently happening only via CoreML (macOS only).
The Reality Check:
While the productivity boost is real, the code is not yet up to my standards. Despite ongoing reviews, I’ve had to accept a certain amount of "slop" that I’ll need to clean up later. More importantly, I hit several conceptual bottlenecks: architectural problems the AI was not only unable to solve without guidance but was unable to even identify in the first place.You can read the full breakdown of the first week and a half (contains links to code and various illustrative commits) in the link for this post.
And the backstory on why I'm doing this is at: https://medium.com/@polyglot_factotum/the-slop-diaries-imple...
I’d be happy to hear from others on how they balance the "syntax speed" of AI with the "architectural integrity" required for long-term projects.
I'm using headless Pi with my own UI and sandbox client, https://github.com/gterzian/uni03C0, as well as a bunch of Pi extensions for things like accessing Web standards and browser use via CDP for testing.
Switching to 4.1 today...
Edit: it seems they pushed the date at which they route the Pro calls to new Flash, so today I ended up paying regular Pro rates thinking I was using the new Flash; an example of how their offering is not quite as predictable as I would like it to be (the other is cache performance being unpredictable).