> But how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?
Given that a small model that was trained only on pre-computer-era texts can write simple python programs after having python explained to it in context I'd say that LLMs must be pretty good at understanding coding.
Given that a small model that was trained only on pre-computer-era texts can write simple python programs after having python explained to it in context I'd say that LLMs must be pretty good at understanding coding.