neerajk
Joined 12 years ago
Karma
102
Submissions
30
Member since
Aug 24
Recent submissions
Comment · 2 months ago
On not re-inventing the wheel obviously.
Comment · 3 months ago
There is a Sanjeev Arora paper "A Theory for Emergence of Complex Skills in
Language Models" (https://arxiv.org/pdf/2307.15936) on this subject. The key idea is there is cross entropy (how "surprised" the model is with the "correct" next token, lower is better), some of which is inherent in the language and therefore unavoidable, and the rest is model error, and that this portion of the cross entropy is reduced with scaling.
And as scaling reduces a model's excess entropy, the model can become good at combinations of skills much faster than you would expect if it had to separately see and memorize every combination. They call this "slingshot generalization".
Comment · 3 months ago
I rarely see gears in these bicycles. Is the idea that should a pelican need to go uphill it could just fly.
Comment · 7 months ago
I'm just getting started ;)
Comment · 10 months ago
In contrast, the 2nd Ave Subway extension here in NY cost $4.5 billion for 2.9 km
Comment · 1 year ago
Am I allowed to stay with a relative or a friend when I visit their town? I am totally ignorant of local norms, maybe I should be in the Holiday Inn next door instead?
Comment · 1 year ago
segfaults raise my belief in spirits
Comment · 1 year ago
"I'm so sick of hearing about bronze. Bronze bronze bronze bronze. Whats wrong with stone? Does stone not work all of a sudden?"
Comment · 2 years ago
Thats when the sea receded, so its "only" 5,000 years old after all.
Comment · 2 years ago
aside: me after reading the headline: Shouldn't that be Octopii?
But no! "The plural form octopii is doubly incorrect. Firstly, octopus derives from Greek, not Latin; its etymologically-consistent plural form is octopodes. Secondly, even if octopus were a second-declension Latin noun, the plural form would be octopi; in the correct plurals radii and gladii, with which octopii is analogous, the first ‘i’s are part of the words’ stems (radi- and gladi-), and not their case endings — for octopii to be the plural, *octopius would need to be the singular."
Thanks wikipedia.
TIL!
Comment · 2 years ago
Exactly. And if you don't say seven in the emphatic manner George says it who are you.
Comment · 2 years ago
We'll give "real
actual bug" a pass :)
Comment · 2 years ago
> The nominal mode enables motion through the water at 3.6 km/h, and for speed-seekers, the SEABIKE can reach a maximum of 7.9 km/h – much faster than normal swimming speeds or even flipper-assisted swimming.
https://www.nauticexpo.com/prod/seabike/product-68606-564117...
Pretty fast, but "superhuman"? For short distances Michael Phelps can swim faster :)
Comment · 2 years ago
it is science that discovers errors in science :)