Comment by UltraSane

I think long term LLMs should directly generate Abstract Syntax Trees. But this is hard now because all the training data is text code.