Isn’t the point of arc agi 3 to measure general game intelligence even if it’s not a AAA game? Frontier models have obliterated that. And they do keep most of it private so no gaming there.
The models these days have 3 components that is for sure going to get better and better
1. Very good vision (it can identify almost anything in a picture)
2. Keyboard and mouse actions (slow but it does it well)
3. General world understanding (this one is toss up but given Astra or Fable, it surely does well even if it takes a while)
Now combine all these with specific RL environments that may use specific games which also generalizes to an extent and we have a system that can only get significantly better from here.
As for speed, I’d say it’s a question of how much money you are willing to throw (like Cerebras for example).
The models these days have 3 components that is for sure going to get better and better
1. Very good vision (it can identify almost anything in a picture) 2. Keyboard and mouse actions (slow but it does it well) 3. General world understanding (this one is toss up but given Astra or Fable, it surely does well even if it takes a while)
Now combine all these with specific RL environments that may use specific games which also generalizes to an extent and we have a system that can only get significantly better from here.
As for speed, I’d say it’s a question of how much money you are willing to throw (like Cerebras for example).