sl8r
Joined 12 years ago
Joined 12 years ago
Thanks for reading. The 2D simulation section might be more interesting on a first read — it makes the math less mysterious, I hope!
Theta represents all the model params — all the weights in the neural network. The convention is to write theta for the “learned” score function and omit theta for the “true” score function.
I personally find the SDEs the most intuitive, and the deterministic ODE / consistency models / rectified flow stuff as ideas that are easier to understand after the SDEs. But not everyone agrees!
In the “Sad Irons” chapter of Caro’s LBJ biography, he talks about the pre-electrification lives of Texas farmers. In comparison with that, our whole day is leisure.
Similarly: As late as 1900, the poor in Europe were so severely malnourished that growth stunting was common. Look at Our World in Data’s charts of height over time. Or Robert Fogel’s “The Escape from Hunger and Premature Death.”
Etc. Etc.
credit bubbles often pop when some institution can't cover its obligations, but asset bubbles don't seem to need such a failure — and can deflate on their own.
Other references I found interesting:
- Cover and Thomas's "Elements of Information Theory" shows some interesting connections between Kelly betting and optimal message encoding.
- Ed Thorp, the inventor of card counting, has a nice compendium of papers on this in "The Kelly Capital Growth Investment Criterion".E.g., say that the random numbers are uniformly distributed between -200 and 200. If somebody says a number like 425, then I know they weigh at least 225 pounds. And the probability they weigh more than 225 - k pounds is 1 - k/400.
* "hard landing" makes the airline industry seem tumultuous and exciting
* "a man for all markets" is the autobiography of ed thorpe, father of card counting and quant hedge funds
* "unix: a history and a memoir" is a mischievous first hand account from brian kernighan (of unix / c / awk fame)
I wouldn't say that you can't do this stuff without category theory, but it does make it easier / clearer. (Similarly, you can do a lot of geometry without coordinates, but coordinates definitely make some stuff easier / clearer.)
[1] https://www.wikiwand.com/en/Brouwer_fixed-point_theorem#/A_p... [2] https://www.wikiwand.com/en/Eilenberg%E2%80%93MacLane_space
I too found it really surprising that the reverse-time equation has a simple closed form. Like, surely breaking a glass is easier than unbreaking it? That’s part of what got me interested in this stuff in the first place!
If you haven’t seen it yet, highly recommend the blogs of Sander Dieleman & Yang Song (who co-invented the SDE interpretation).