I did somewhen toy with the idea of comparing how a line of code is written to what an LLM-based next-token-prediction would emit for the same line. Then it might be possible to highlight the pieces of code that most differ from what an LLM would have written.
The base form isn't too useful by itself, but I expect if one would "seed" the LLM with a prompt about focusing specifically on a particular class of errors, it might be possible to find violations by seeing where the code completely deviates from what an LLM would have written.
Never got around to doing anything with this, but I expect a prototype should take no more than a day with some agent-assisted coding ;) Tell me how it went if you try!
The base form isn't too useful by itself, but I expect if one would "seed" the LLM with a prompt about focusing specifically on a particular class of errors, it might be possible to find violations by seeing where the code completely deviates from what an LLM would have written.
Never got around to doing anything with this, but I expect a prototype should take no more than a day with some agent-assisted coding ;) Tell me how it went if you try!