The "it's been done before" one is pretty relevant. It means the model didn't actually generate the game, but likely pulled it more or less straight out of its training data. It's still very cool that you can ask it for something and it can basically mine the entire (2021) internet for it, but it's not the same as being able to create something really new.
I've noticed the same thing testing it on various coding questions. It's extremely good at problems that have solutions online. And given stackoverflow, that's a lot of problems. If you manage to hit it with something that it hasn't seen before though, even if it's conceptually very straightforward, it tends to just generate a mix of boilerplate and nonsense.
I've noticed the same thing testing it on various coding questions. It's extremely good at problems that have solutions online. And given stackoverflow, that's a lot of problems. If you manage to hit it with something that it hasn't seen before though, even if it's conceptually very straightforward, it tends to just generate a mix of boilerplate and nonsense.