1. Capturing the entire scene with photos or video and converting ("training") these into a 3d Gaussian splat. This is what Vincent did so well with Grace. Open source tools for training are Brush and Lichtfeld Studio.
2. Creating a user experience with the resulting capture. In the case of Grace it involved adding the peek effect, camera controls, moving cars etc. This was all done using PlayCanvas as the article describes, which is also open source.
There is still some art to doing 1 well, but with a capable LLM agent and our (PlayCanvas') awesome community on Discord, you could probably manage both.
1. Capturing the entire scene with photos or video and converting ("training") these into a 3d Gaussian splat. This is what Vincent did so well with Grace. Open source tools for training are Brush and Lichtfeld Studio.
2. Creating a user experience with the resulting capture. In the case of Grace it involved adding the peek effect, camera controls, moving cars etc. This was all done using PlayCanvas as the article describes, which is also open source.
There is still some art to doing 1 well, but with a capable LLM agent and our (PlayCanvas') awesome community on Discord, you could probably manage both.