raindear

There has been a slow progress in computer vision in the last ~5 years. We are still not close to human performance. This is in contrast to language understanding which has been solved - LLMs understand text on a human level (even if they have other limitations). But vision isn't solved. Foundation models struggle to segment some objects, they don't generalize to domains such as scientific images, etc. I wonder what's missing with models. We have enough data in videos. Is it compute? Is the task not informative enough? Do we need agency in 3D?

Are dead ReLUs still a pronlem today? Why not?

How far can you go by improving the curriculum? Start simple. Find a shorter and shorter sequence of examples that gives you thd best result. What is the shortest sequence to get to some perplexity? Why?

Progress is what happens thanks to AI skeptics busy defining model limitations. The limitations set attractive bars to pass.

I read that deepseek was trained on western llm output. So it is expected to have the same biases.