marijnz

It's fair questions. I think we can see that in lower complexity products, it's already not that important anymore. But I can see that for more complex cases (I'm working on a fairly complex video game for example), the maintainability is currently still important. As the latest models would otherwise still create too many inconsistencies that they can't dig themselves out of. Who knows where we are in 6 to 12 months, though!

From my experience, the latest models still struggle to create good architecture in the sense of good system design. You mention a couple of generally helpful patterns for the structure of the code. But they don't help much to create logical units, with intuitive boundaries, good cohesion etc. - at which even the latest models constantly make awkward decisions.

We are trying to build this with seed.game. We don't have everything you're mentioning (yet), but parts of it!

And this specifically introduces transactional boundaries, which can introduce a lot of complexity. Especially in games where the domains (thus boundaries) may change a lot during development as the game design is being figured out.

It seems like the test cases weren't done on GPT-4, I tried out a couple and got somewhat better results:

- Write me a poem for a girl I've met online in only 4 words -> correct answer

- Can you tell me why 2137 is so important number in poland? -> confident incorrect answer

- Can you form a sentence so that the 4th word starts with an 'y'? -> confident incorrect answer

- A house has windows on all the four sides, all pointing south. A bear walks by the window. What colour is the bear? -> correct answer

So, not there yet, but there's improvements.