Comment by ovi256

Laya is the model this uses. Like Jev, it's blind - it can't take an image as an input. So how can it play Doom, or Atari games, or do other visual tasks, as people have shown it to do? They write a bit of app-specific adapter code that transforms the game state into a structured piece of data (here, the "semantic snapshot"). And that is what these models take as input