Comment by sheepseb

No, not shadows. I use the heading as a token, in Doom the minimap has a player token and the facing (the current 3D view), I tried using only the frame but it was harder to learn only from the wall patterns. So the agent isn't magically grokking position, its using the map. As for the Pac Man, it's based on mouth and has tokens for "keep going" or N/E/S/W direction. For breakout it uses 4 ball tokens, one per diagonal and it moves one cell per step in that diagonal, and for pong the ball has 6 tokens (combiantion between [up, level, down] with [left, right], e.g up right, level down)