pash

Most predictive tools in economics and finance have “surprisingly” little economic content; but once you understand the distinction between the two goals, it should be unsurprising that predictive models tend to make few economic assumptions, relying rather on general statistical techiniques or on econometrics that incorporate a minimum of theory [1]. From that understanding comes the humbling realization that predicting the future is quite difficult in a context in which the relevant processes are continually seeking an equilibrium that often implies unpredictability. [2]

I’m not an economist, but I do a lot of applied financial-economic modeling. State-of-the-art LLMs are really, really terrible at economic intuition. They will hinder, not help, in formulating an economic model, which is a process of coming up with a set of modeling assumptions that lead to a useful (implicitly, tractable) model. LLMs are, however, quite good at math, and I’ve found them very useful in iterating through different sets of modeling assumptions to identify those that lead somewhere useful. Not having to work out all of the mathematical details myself, and thereby avoiding getting lost in the weeds and being better able to maintain a higher-level perspective on what I’m trying to accomplish, has accelerated my work immensely. But it’s a process of leading the LLM by the nose the whole time and asking it to fill in the details.

I should note, thought, that if you indotend “AI” to mean more than LLMs, them yes, there is starting to be a lot of good work done on predictive economic models that use specialized neural networks as black-box functions to compute model quantities that are otherwise difficult to come up with, just as is also happening in applied physics and other fields.

Edit: fergal_reid’s reply below is correct. Porter did serve several years at the start of his career as a pilot, in the early 1930s, before spending the bulk of it as a signals officer. Holden also had his wings, but never served as a pilot. I read Porter’s comment as a joke between two RAF officers who were not “really” pilots, but now I’m not sure how to read it.

After seeing AlexNet’s results, all of the major ML imaging labs switched to deep CNNs, and other approaches almost completely disappeared from SOTA imaging competitions. Over the next few years, deep neural networks took over in other ML domains as well.

The conventional wisdom is that it was the combination of (1) exponentially more compute than in earlier eras with (2) exponentially larger, high-quality datasets (e.g., the curated and hand-labeled ImageNet set) that finally allowed deep neural networks to shine.

The development of “attention” was particularly valuable in learning complex relationships among somewhat freely ordered sequential data like text, but I think most ML people now think of neural-network architectures as being, essentially, choices of tradeoffs that facilitate learning in one context or another when data and compute are in short supply, but not as being fundamental to learning. The “bitter lesson” [1] is that more compute and more data eventually beats better models that don’t scale.

Consider this: humans have on the order of 10^11 neurons in their body, dogs have 10^9, and mice have 10^7. What jumps out at me about those numbers is that they’re all big. Even a mouse needs hundreds of millions of neurons to do what a mouse does.

Intelligence, even of a limited sort, seems to emerge only after crossing a high threshold of compute capacity. Probably this has to do with the need for a lot of parameters to deal with the intrinsic complexity of a complex learning environment. (Mice and men both exist in the same physical reality.)

On the other hand, we know many simple techniques with low parameter counts that work well (or are even proved to be optimal) on simple or stylized problems. “Learning” and “intelligence”, in the way we use the words, tends to imply a complex environment, and complexity by its nature requires a large number of parameters to model.