For anything where "the answer" wasn't already in the training data you just get some arbitrary correlated tokens out, as that's all a LLM can do. Of course the meaning of these tokens is just random.
(And even for things that were in the training data you don't have any guaranty they will be reproduced correctly, there is just some chance something meaningful comes out, or it doesn't, it's random.)
The whole idea to use a next token predictor as "answer machine" is completely flawed. This can't work like advertised, and that's by construction.
(And even for things that were in the training data you don't have any guaranty they will be reproduced correctly, there is just some chance something meaningful comes out, or it doesn't, it's random.)
The whole idea to use a next token predictor as "answer machine" is completely flawed. This can't work like advertised, and that's by construction.