I know less about this than every other commenter here, but both weather patterns and market performance do seem predictable based on past behavior when modeled at the right level of abstraction. “Sunshine on Monday” does not imply “rain on Tuesday”, but “cold front moving in Monday night” does. (Likewise “stock up Monday” doesn’t imply “stock down Tuesday” but “CEO arrested for fraud on Monday” does.)
I think this is relevant to the discourse on LLMs/programming because for months, people said “they’re just regurgitating their training set,” but now I think people are seeing (I am seeing) that they do learn more abstract models of the world than that. I don’t really know how, but it’s why they can generalize from other codebases and tools and so on.
I think this is relevant to the discourse on LLMs/programming because for months, people said “they’re just regurgitating their training set,” but now I think people are seeing (I am seeing) that they do learn more abstract models of the world than that. I don’t really know how, but it’s why they can generalize from other codebases and tools and so on.