jballanc
Joined 18 years ago
Karma
9,437
Submissions
1,596
Member since
Nov 23
Recent submissions
Comment · 5 days ago
When I was in the market for a new machine, I had the same attitude. As I was about to hit "Buy" on a standard issue $2k MBP, my wife said, "What if we got you a top of the line machine?"
I replied that it would make no sense unless I was going to start running local inference or training my own models locally...
2 years later, and this MacBook Pro M4 Max w/48GB is the best $4k I've ever spent!
Comment · 6 days ago
...that's...not how exponents work
Comment · 20 days ago
While poetic, I suspect that the utility of including this in the prompt is much more pragmatic: models refined on datasets of turn-based interactions will have weights biased toward checking in with the "user". Saying "believe in yourself" probably introduces just enough of a counter-bias to push the model into a local basin of deeper thought.
Comment · 20 days ago
Yes, Tylenol's molecular method of action has never been experimentally confirmed.
Comment · 20 days ago
I have no particular dog in this fight, but I thought it might be pertinent to point out that we still don't know the exact mechanism behind why Tylenol works...
Comment · 25 days ago
In graduate school he had a reputation for being the best at synthesizing LSD.
Comment · 25 days ago
If you think Kekule's story is wild, you should read about how Kary Mullis came up with the idea for the polymerase chain reaction!
Comment · 2 months ago
terminal-code has been replaced by a shell script which 1) Generates a syslog message at level LOG_EMERG; 2) reduces the user's disk quota by 100K; and 3) RUNS ED!!!!!!
(adapted from the venerable "Ed, man! !man ed": https://www.gnu.org/fun/jokes/ed-msg.html)
Comment · 3 months ago
When I was in graduate school, I was doing a lot of molecular dynamics simulations, but I did not have access to a lot of high-performance hardware. The math department had a 4-node Cray that was impossible to reserve time on, and the CS department had a 16-node Dell cluster that ran SGE and was decently reliable. When those resources ended up not being enough for my crazy array of tasks, I got resourceful and started gathering every single outdated CPU anyone anywhere on campus was getting rid of or had no use for, and built my own cluster in our lab. Before finishing my Ph.D. I had 12 SGI Indys, 8 PowerMac G3 6300s, 10 PowerMac G3 7600s, and 20+ Dell P3 towers in my "cluster".
NetBSD is the only thing that made that possible.
Comment · 3 months ago
This is almost certainly what it was. Wiki even has a photo of the USS Gerald Ford undergoing blast tests off of Ponce Inlet, and mentions that it registered as a M3.9 quake: https://en.wikipedia.org/wiki/USS_Gerald_R._Ford#Operational...
If I had to guess, this is probably the USS John F. Kennedy: https://en.wikipedia.org/wiki/USS_John_F._Kennedy_(CVN-79)
Comment · 5 months ago
I've been working on RVW, my adaptation of the standard transformer model that is capable of online continual learning without catastrophic forgetting. I finally published the first pre-print of my early experiments: https://doi.org/10.5281/zenodo.20064617
Now I'm working on expanding the work into more parameters and improving performance. I just finished an extremely harsh test of a Nemotron-flavored RVW that consisted of stretches of a random assortment of domains interspersed with long runs of single domains. Across all of it the model didn't forget (and actually improved on some of the more challenging domains). PPL on SmolTalk is still in the ~18 range, which I'd like to get lower, but this is all with only 4B params.
Currently, I'm training a Llama 3.2-flavored RVW with only about 2B params to see how that turns out. Depending on results of that, I may take it to Gemma 4 next.
Get in touch! [email protected] https://github.com/jballanc
[ my public key: https://keybase.io/jballanc; my proof: https://keybase.io/jballanc/sigs/pxb6FPsVF9hoQMyb2ap0euWoNG3YxwN2g4VY94RRxX8 ]