TobiasEnholmX
Joined 3 years ago
Joined 3 years ago
For example, you can attach a LoRA specifically to one object and run a separate workflow just for that, giving you way more control. That’s a big difference—Gaussian Splatting doesn’t naturally lend itself to object-level edits since everything is blended into the same representation.
By passing a structured 3D-rendered image to the AI, you increase the chances of the video model generating exactly what you want, rather than relying on unpredictable, frame-by-frame AI improvisation.
1. I assume you mean patience when setting up a 3D scene? That’s definitely a factor, but it’s getting easier with image-to-3D tools, and AI can even assist with object placement to speed things up.
2.Yeah, predictability is key. Our approach is about making it easier to generate high-quality, consistent images, which are then fed into video models—rather than relying on direct video-to-video style transfer, which can be more chaotic.
3. Agreed! AI can animate things that traditional methods struggle with, but consistency is still a challenge. This workflow helps strike a balance between AI flexibility and user control.
Your experience with Minimax sounds cool! Adding 'animated' to the prompt helping consistency makes sense—AI models often struggle with structure, so any guidance helps.
Overall, it worked. No sudden changes in proportions, clothing, or style. Still, there are some limitations, especially with fine details.
We’re looking into whether this could be useful as a tool and would love to hear what you think: Has anyone experimented with 3D + AI generation for images or video? or sees a better way to approach this?
Demo and details in the blog: https://backdroptech.github.io/3d-to-video/