thenickdude
Joined 13 years ago
Karma
282
Submissions
172
Member since
Apr 16
Joined 13 years ago
The "biggest problem" being the one thing that is trivial to verify against concrete databases is a bit convenient don't you think?
I think it's more likely that it makes mistakes evenly but the one thing that you are able to check with certainty is the only place you discover the errors.
https://medium.com/@TalBeerySec/a-short-note-on-aws-key-id-f...
It turned out that the LLM-runner's default setting for handling queries that were too long was to silently delete the middle of the query. So the LLM saw the beginning of my ask, and the tail end of the code context, and nothing else.
https://i.imgur.com/LRotRlS.png
The link goes to a typo-squatted domain using an "l" instead of an "i".