mattmight
Joined 16 years ago
Karma
2,847
Submissions
166
Member since
Jun 28
Recent submissions
Comment · 5 months ago
Original article author here.
One of the interesting things about the lambda calculus is its universality: by itself, it's a complete foundation for computation.
Here's a different old post of mine showing how to build the rest of the programming language, all in a miniscule subset of Python that is the pure lambda calculus:
https://matt.might.net/articles/python-church-y-combinator/
You can even extract recursion out of the Y combinator or the more primitive U combinator -- out of nothing but lambdas!
So, it's lambdas all the way down.
Another interesting thing about the lambda calculus is that it wasn't intended to be a programming language. When Alonzo Church created it, there were no computers to program.
Alonzo Church was trying to solve problems in the foundations of mathematics.
But, untyped lambda calculus has a "bug" that makes it problematic for mathematics -- the self application that enables recursion is a problem if you're a logician who cares about soundness, but it's fantastic if you're a programmer.
I don't think of functional languages as obfuscating. I think of them as terse and expressive. They let me most directly encode the model in my head as running code.
Comment · 5 months ago
Good point -- JavaScript's syntax grew up since I wrote that article!
Comment · 5 months ago
Thank you for the kind words! I keep hoping I can find the time to do more writing again. And, yes, I started that blog in 2008.
Comment · 6 months ago
Wondering the same, and in somewhat different terms.
And as models shrink in size yet go up in intelligence and performance, I'm finding ever more life in older hardware.
When I got my M1 Max in 2021, GPT-3 was about 1.5 years old and it was SOTA.
Yet, that machine is now able to run models that crush with gpt4, and even compete with o1 (SOTA from about 1.5 years ago.)
The idea that I could run something like that locally would have seemed absurd in 2021.
Yet, if somehow I'd had those local models in 2021 on the exact same hardware, I would have had, by far, the most powerful AI on the planet -- and that would have remained true for the next several years.
I'm also noticing that the ever-improving smaller models I can run on this machine are crossing the "good enough" threshold for ever more tasks by the month.
I just don't need a frontier model for every task.
I have an M4 Max 128 GB RAM now, but I still find plenty of tasks to delegate to the M1 Max machine.
I don't know how far this can go in the limit in terms of packing more intelligence into smaller models, but older hardware, if maintained well, seems like it's going to increase the value it can deliver in terms of "intelligence per watt-hour."
Comment · 9 months ago
Blog: https://matt.might.net/articles/
Main site: https://matt.might.net/
Comment · 1 year ago
Nothing to feel bad about. Thank you for sharing that too.
My son’s life changed my own in profound ways, and even though he died four years ago, he is still changing my life in profound ways. I am always grateful for the reminder and to reconnect with the purpose that his life gave to mine.
That post also reminds me that while he was alive, I did the best I could for him under my abilities, and that’s all any parent can do in the end.
If you want to know more about his life, I wrote on it here: https://bertrand.might.net/
Comment · 1 year ago
Happy to hear that! Thank you for sharing it with others. :)
Comment · 1 year ago
So happy it helped! Thank you for the kind words!
Comment · 1 year ago
Good for you!
Comment · 1 year ago
Original author of the guide here. Wonderful to see these little illustrations still making the rounds. I first published them in 2010!
To those in the comments who mentioned you are just starting your own PhD: Good luck to you! And, I hope you, like I once did, find a problem that you can fall in love with for a few years.
To those just finished: Congratulations! Don’t forget to keep pushing!
To those many years out: You have to keep pushing too, but there can be tremendous value in starting all over again by pushing in a different direction. You have no idea what you may find between the tips of two fields.
Comment · 2 years ago
"Think of it as an unholy combination of bash and Lisp" reminded me of Scheme Shell: https://scsh.net/
My Ph.D. advisor Olin Shivers created this. It's more pulling bash into Lisp than Lisp into bash.
It doesn't seem to be maintained anymore, but some of its ideas live on in Racket.
Comment · 2 years ago
Same! I use ollama a lot, but when I need to do real engineering with language models, I end up having to go back to llama.cpp because I need grammar-constrained generation to get most models to behave reasonably. They just don't follow instructions well enough without it.
Comment · 3 years ago
Glad to see you're still doing the right thing, JT!
Comment · 3 years ago
(Article author here.)
This was the reference I used for TeX programming: https://pgfplots.sourceforge.net/TeX-programming-notes.pdf
Comment · 3 years ago
I had a similar thought after finishing:if you wanted to really learn a new language in depth, you could solve all 25 days using that language.
Home page: http://matt.might.net/ Blog: http://matt.might.net/articles/