Comment by bberrry

Do LLMs really encode knowledge regardless of language, and tap that knowledge irrespective of which language you query in?

My mental model of LLMs being next-word predictors with a long context window suggests they don't. Are there any papers on this?