Very interesting outcome. Shows how far it’s possible to get without being idiomatic.
You say the docs contain guidance on structuring, high-level constructs, and how to make things idiomatic. It would be an interesting test to hand the unfixed revision of the code to an LLM while also giving it the docs, and say “make any fixes to make this conform to standards of the framework and libraries”.
If it picks up the same things that’s great news for novice programmers and anyone new to a framework!
LLMs will improve at making these fixes over time - even if they’re currently bad at it that won’t last.
You say the docs contain guidance on structuring, high-level constructs, and how to make things idiomatic. It would be an interesting test to hand the unfixed revision of the code to an LLM while also giving it the docs, and say “make any fixes to make this conform to standards of the framework and libraries”.
If it picks up the same things that’s great news for novice programmers and anyone new to a framework!
LLMs will improve at making these fixes over time - even if they’re currently bad at it that won’t last.