Isn't it entirely possible that they did exclude GPL licensed code, but somebody somewhere has violated copyright and copy-pasted that snippet into non-GPL-licensed code that they trained on?
They could try to trace every single code snippet they train on to its "true source" and use the license for that, but that's not very well-defined, and is a lot harder, and it's never going to be 100%.
They could try to trace every single code snippet they train on to its "true source" and use the license for that, but that's not very well-defined, and is a lot harder, and it's never going to be 100%.