Yeah, I love this. We can't manage what we don't measure, and adjusting the mesurement to be on the abstract syntax level removes an entire layer of "mess" that comes from comments and formatting.
I'll bookmark it and probably adjust the measurement methods for future review purposes. I've often found my LLM workflows to produce inane and overly complex code, and allowing me to see a single figure will allow me to more easily eval generated code.
I'd love to hear people's lived experience, though. I think heuristics in this space still allow for a lot of optimizations of code generation and pipelining.
I'll bookmark it and probably adjust the measurement methods for future review purposes. I've often found my LLM workflows to produce inane and overly complex code, and allowing me to see a single figure will allow me to more easily eval generated code.
I'd love to hear people's lived experience, though. I think heuristics in this space still allow for a lot of optimizations of code generation and pipelining.