duckworthd

Thank you :)

I hope so, but it'll be a good while before we can release anything. We have tight dependencies to other not-yet-OSS libraries, and until they're released, our won't work either.

Maybe! That's the seed of a completely different research paper :)

The big difference is access to fisheye lenses a burst mode that can be run for minutes at a time, and the ability to minimize the amount of camera post processing. In principle, the capture could be done with a smartphone, but the experience of doing so is pretty time consuming right now.

You're looking at something called a "neural radiance field" backed by a sparse, low resolution voxel grid and a dense high resolution triplane grid. That's a bit of a word soup, but you can think of it like a glowing fog rendered with ray marching.

The benchmark details are a bit complicated. Check out the technical paper's experiment section for the nitty gritty details.

What's worked well for me: Find a way to put what AI/ML on your critical path. Think of it like learning a new language: classes, lessons, and watching TV helps, but nothing works like full-on immersion. In the context of AI/ML, that means find a way to turn AI/ML into your full-time job or school. It's not easy! But if you do, you'll see endless returns.

If you don't have a solid enough footing to get a job in the field yet, the next best thing in my opinion: find a passion project and keep cooking up new ways to tackle it. On the way to solving your problem, you'll undoubtedly begin absorbing the tools of the trade.

Lastly, consider going back to school (a Bachelor's or Master's, perhaps?). It'll take far more than 1 hour/day, but I promise you, you'll see results far faster and far more concretely than any other learning strategy.

Good luck!

Context: I've been a Researcher/Engineer at Google DeepMind (formerly Google Brain) for the last ~7 years. I studied AI/ML in my BS and MS, but burnt out of a PhD before publishing my first paper. Now I do AI/ML research as a day job.

This should absolutely be possible! The hard part is making it look natural: NeRF models (including SMERF) have no explicit materials or lighting. That means that any character inserted into the game will look out of place.

In terms of the live viewer, there's actually no limit on footprint size. 300 m^2 is simply the biggest indoor capture we had!

The viewer biases towards assets closer to user's camera (otherwise you'd have to load the whole scene!). We tried training SMERF with a larger number of smaller submodels, but at some point, it becomes too onerous to train and quality begins to suffer.

Thanks for the feedback!

I agree, we could do better with the movement UX. A challenge for another day.