What LLMs learn is a "conditional probability distribution" that is the probability that a particular word is used contingent on the
other words used in the text.
If it learns to write in the style of Donald Trump that's because it sees "in the style of Donald Trump" in the prompt and that elicits it consider what word Donald Trump would use in a situation. If it is asked to "talk like a pirate", it similarly is drawing from a different probability distribution.
It doesn't literally memorize the content of all the books it was fed because it does not have enough neural connections to encode it all without loss. What's miraculous about it is that it models these "conditional probability distributions" very effectively, it doesn't need to see an example of seeing absolutely everything written in "talk like a pirate style", it can somehow blend what it knows about the conditional probabilities of pirate talk and the conditional probabilities of other conversation and frequently produce a convincing result.
So it is very possible it can follow patterns and make inferences based on what it "learned" in pre-training and what it has just been told and come to conclusions that weren't already fed into it.
On the other hand if I want to talk with it about football it is not at all qualified to have an opinion about "Do you think Arsenal or Man City will win the championship this year?" because the LLM doesn't contain any knowledge about this season at all. (Note if you ask Bing Chat it does a search lookup and interprets the results.)
Also note it is all statistical so it is likely to "make stuff up" whether it is facts that it could have read in a book or when you ask it to make some conclusion based on the knowledge available to it. It can very well make up something that wasn't fed it into it originally, whether or not it is right is a roll the dice sort of thing.
If it learns to write in the style of Donald Trump that's because it sees "in the style of Donald Trump" in the prompt and that elicits it consider what word Donald Trump would use in a situation. If it is asked to "talk like a pirate", it similarly is drawing from a different probability distribution.
It doesn't literally memorize the content of all the books it was fed because it does not have enough neural connections to encode it all without loss. What's miraculous about it is that it models these "conditional probability distributions" very effectively, it doesn't need to see an example of seeing absolutely everything written in "talk like a pirate style", it can somehow blend what it knows about the conditional probabilities of pirate talk and the conditional probabilities of other conversation and frequently produce a convincing result.
So it is very possible it can follow patterns and make inferences based on what it "learned" in pre-training and what it has just been told and come to conclusions that weren't already fed into it.
On the other hand if I want to talk with it about football it is not at all qualified to have an opinion about "Do you think Arsenal or Man City will win the championship this year?" because the LLM doesn't contain any knowledge about this season at all. (Note if you ask Bing Chat it does a search lookup and interprets the results.)
Also note it is all statistical so it is likely to "make stuff up" whether it is facts that it could have read in a book or when you ask it to make some conclusion based on the knowledge available to it. It can very well make up something that wasn't fed it into it originally, whether or not it is right is a roll the dice sort of thing.