lishali88

This post is actually about not making the same arcade games and making a 3D Dune Minecraft instead.

I think all good apps should strive to achieve a flow-state, and Rosebud is no exception; we aspire to achieve the same. The AI part is to solve a problem rather be a feature. The kind of users we want to serve, and what they want, is to make games. The chat interface is a way to create a flexible interface to do that, not force AI into the solution. "AI build me ___" is only addressing simple asset generation or addition of game features, which is a very limited use of AI to solve the bigger problem we are focused on, which is to decrease the barriers to entry to making games.

There is also a useful distinction between generative AI features of gameplay versus AI in aiding the creator (game dev) to make games. We do both, but they are driven by different user needs. For instance, for the former, there are AI powered characters which gives players a sense of spontaneity in the gameplay.

Thanks for stopping by! That was a fun video to be a part of, Grant’s math videos in general are so well done. I think this video was from 2017. Funny thing is that even back in 2014/2015, which is when I officially changed my PhD from pure math to AI, I had the fear that it was a little late (was originally from Toronto where neural nets were worked on my Hinton and his students —-Ilya, James…etc….so I felt like I had my chance to join much earlier and missed it). Only after 2012 did we start to see super human performance of AI in some tasks (namely, at first, classification). It was the Lee Sedol match with AlphaGo that convinced me that the field became a lot more interesting after I got to Berkeley (and also the generative stuff coming out early days from Alyosha’s lab and Deep RL stuff from Pieter’s).

But look at the world now! It’s never too late, jump in and build if you are excited now!

Thanks for summarizing these and spending a few hours trying out Rosebud!! Some detailed responses below:

"I found getting assets into the game a bit tricky. Sometimes it would try updating the code to reference assets that we hadn't created yet. After that I couldn't get it to generate assets for me."

This is an known issue we are trying to solve. But to give more color on why it's been tricky, since we are relying on a chat interface, our agent has properly determine is your ask includes an ask for asset generation, and if so, it has to determine whether an asset exists, and if it does not, whether to generate an asset, after which it needs to determine where to insert it and whether to modify other parts of the code when inserting. For instance, in the character templates, the AI can successfully modify an asset and character's description in one prompt, but sometimes it doesn't correctly interpret the instruction and pulls existing assets from the Phaser JS library. In short, it's quite finicky to get agents in prod to behave reliably 100% of the time. But we are working on it! Have many ideas of how to improve and are experimenting.

"I thought this would be a useful tool for prototyping, but as a programmer I would probably never use this to release a finished product. Maybe it would be better to target this kind of tool for prototyping instead of being able to make a full game that people will want to pay money for." & "I tried making a simple turn-based (american) football game. It's pretty good for getting some boilerplate set up, but sometimes it felt like it would have been easier to modify the code myself at times than try to explain what I wanted modified to the AI."

For users who already know how to develop, we've definitely heard similar validation that this is better for prototyping currently than for final games. I think this stems from the fact that 1) currently we don't have multi-file support and more advanced features like multiplayer yet, which makes it hard to compare with what's achievable by a more advanced game dev. And 2) the chat based approach is not always reliable, so it's frustrating sometimes to get what you want versus coding it up yourself. This is why we are also targeting less technical users, because they don't have many options available to them that can be as flexible as Rosebud. We have been delighted to see what these users create because some of the stuff gets very creative and addictive, even with our current limitations.

"It seems like whenever I did try to modify the code by hand, my changes would get wiped out by the next iteration generated by the AI. Maybe it was referencing the old code?"

This is a bug. If you don't mind sharing in the feedback channel specific cases when it does this, that would be super helpful. We've been fixing aspects of this bug over the last few week.

"Sometimes the AI would get stuck on something (e.g. generating an asset), or it would make a mistake. In that case I couldn't figure out a way to "go back" to a previous iteration of the game."

We have a history feature we are shipping in stages. Right now you can only regenerate, but we want to let users go back to different states of their game. Many users have wanted this feature and we just have to make some choices about what gets saved in the history (all llm changes or also manual, ... etc).

"At one point I tried to get the AI to make a change, which it did, but the code it generated had a large portion of the original code based removed and replaced with something like `// The rest of your code here.` which obviously broke the game."

This stems from a context length issue and is definitely annoying. We also have plans to implement diffs and other solutions that should let users run into this issue less.

Do you use blender and what would you want AI to help in that workflow?

Thanks for trying it! One of the more impressive things we have seen so far is how some beta testers with no coding background were able to get deep into debugging by having our AI explain what the issues are and how to fix them. However, as you pointed out, it does not reliably provide working code all the time for the user to apply directly. This is something we are aware of and are trying to make the generated code better and more reliably provide working solutions with no formatting errors. It is basically quite a hard (and thus fun technical) problem we are solving.

These are great questions!!

Our aim is to make development on Rosebud as easy as possible, which means we are handling various external integrations that are being used by the platform. With that said, some devs may want the flexibility to change what models are used, including switching out internal models we developed with various external ones. If that is true, we’d want to support devs providing their own keys, but only for added flexibility, not as a default. Put it simply, if you don't want to be forced to provide your own keys, we got you covered.

Right now we will try our best not to charge developers for developing on Rosebud. So we will try to optimize on our backend different ways to control costs. With that said, if we have to create some premium tier because the AI inference gets too high to support, we will likely provide options to reduce inference costs in various ways (such as running smaller models for different interactive generative features of a game).

We’d want to provide as much support as possible so that our devs are successful. So if something is not working, let us know! Our devs succeeding is a necessary pre-requisite for our platform succeeding.

See my comment above for another thread and the subsequent comments: "My take is that whenever we can break down barriers to allow more creators to enter, it's ultimately better for the entire industry. As a platform we can work on curating and filtering for high quality game content from our users, but that's a problem of curation rather than artificially limiting creation for fear of people making low quality content. Most content quality like many things are power law distributed anyways."

My optimism is mostly for increasing the number of creators of games --> best games get even better, at the cost of introducing more games, some of which may not be good. Marketplaces being the winners is a symptom of the players being the ultimate beneficiary of increased competition (and thus a surplus of both good and bad games).

I actually don't see this as an AI to replace, but AI to enable more people to create games. So ultimately, it still expresses the desire of the creator, who is human.

We are happy our custom image models are liked by our users and I think helping game devs make good game assets/art is a valuable problem to solve, but not one we are focused on. The first 3-4 years of Rosebud we focused on image generation based app, so I have thought a lot about the opportunity of focusing on a image/asset gen as the main feature of the product. I would argue that it generally has a thinner moat with more capped upside and can more easily be toppled by foundational model companies like OpenAI (with the exception of Midjourney, which is amazing because they have the moat of a community that guides their stronger aesthetics). Furthermore there are a number of other services that focus just on asset gen, and we allow uploads (from assets generated elsewhere) in addition to our own custom models for asset generation for that reason.

The opportunity I am excited most about, and what Rosebud is focused on, is to increase the number of creators of games by many orders of magnitude. That is why we are leaning into an llm native approach for game dev. I also know that the current best models for code gen will be improved dramatically in the next few years and that will have have an even greater impact on the consumer behaviour of who gets to make games and who gets to build software in general. Building a platform around game creation, not just asset gen, will be able to absorb this impending tidal wave of change that I don’t think even incumbent developer tools like Unity or Unreal can address as fast as a startup such as ourselves. The entire game creation work flow is going to get much more intuitive, faster and ultimately be able to generate the quality of games to compete with AAA. Now it is not there, but soon it will be. The game genres we support may look more opinionated and constrained now (ai characters, RPGs, some 3D) but that’s the first step in being able to let our agent based code gen platform perform well in prod. We are making choices that allow us to absorb llm advances later and that generalize well once those improvements happen.

Also, on the IP point, once we let people monetize on the platform, we will be much more strict about what gets to be monetized (i.e only things where it’s ok from an IP point of view).

My take is that whenever we can break down barriers to allow more creators to enter, it's ultimately better for the entire industry. As a platform we can work on curating and filtering for high quality game content from our users, but that's a problem of curation rather than artificially limiting creation for fear of people making low quality content. Most content quality like many things are power law distributed anyways.