jlas
Joined 14 years ago
Karma
873
Submissions
235
Member since
Sep 23
Recent submissions
Comment · 8 months ago
https://en.wikipedia.org/wiki/Zen_and_the_Art_of_Motorcycle_...
Comment · 1 year ago
Notably the scaling law paper shows result graphs on log-scale
Comment · 1 year ago
Yann LeCun discussed why LLMs are not enough for AGI on Lex Fridman pod: https://youtu.be/5t1vTLU7s40?t=138
Comment · 1 year ago
> (more resources = closer to intelligence)
The scaling law only states that more resources yield lower training loss (https://en.wikipedia.org/wiki/Neural_scaling_law). So for an LLM I guess training loss means its ability to predict the next token.
So maybe the real question is: is next token prediction all you need for intelligence?
Comment · 2 years ago
Seriously though when is HiFi coming out?
Comment · 2 years ago
Coding interviews are a lossy process. Once you realize this the angst of rejection softens and it becomes an almost mechanical effort.
Comment · 2 years ago
Don't overlook sources of anti-productivity which might be easier to measure like the time it takes to build & test code changes.
Comment · 2 years ago
If you're a photographer, the low tech way of doing this is just use a polarizing filter
Comment · 2 years ago
This. As you get into later stages, check sizes become bigger and so does risk aversion + due diligence.
Comment · 2 years ago
Non-zero probability I think, one interesting measure of overfitting I've seen is contamination (where the model has seen the exact questions it is being evaluated on) see stats at https://hitz-zentroa.github.io/lm-contamination/