Can someone ELI5 how LLMs are manipulating environments like this? I have a similar question for when you see the blender examples online.
AFAIU the models are multimodal, so you could screenshot factorio and let it come back with some manipulation like move west 5 tiles, do that and then take another screenshot, but that seems wildly inefficient so I'm sure i'm missing something
AFAIU the models are multimodal, so you could screenshot factorio and let it come back with some manipulation like move west 5 tiles, do that and then take another screenshot, but that seems wildly inefficient so I'm sure i'm missing something