hanseul

Yes, D is depth! Sorry that it wasn't clear. The depth here comes from sensors.

I think there are two missing things here. One is that this is not coming from a specialist camera, but from iphone cameras. Many mobile phones (including some android ones) already have hardware to capture depth. And the next is about color information. This video has no information loss in terms of color compared to ordinary videos. It's just that they are mapped into 3D space. The example you are watching has 720p color resolution.

Thank you, thank you!!

Between this app with RGBD videos and 3D reconstruction, I think in the future, there can be some software turning RGBD videos to reconstructed point cloud/mesh, so that you don't need to run a dedicated 3D reconstruction app at the time recording. Maybe I'll be able to work on it like... 3 years later...?

The codebase was originally written for a HoloLens app, so you sort of correctly guessed where it is coming from! I think this app works the best with AR headsets and it also supports AR headsets through WebXR. It's just that the AR headsets are really hard to find outside of labs...

That sounds like a cool idea! Maybe I can make this website to support phone AR, then you can place the video on top of some ruler or furniture for your measuring purposes? It's nice to hear about potential use cases and good luck with your renovation!

Thanks, I agree that what Apple is doing is pretty cool, both hardware and software-wise. About supporting devices, while the better quality rear depth cameras are only available for 12/13 pro devices, the front depth camera is included in all iphones supporting face ID (3 years old), which is sufficient for running this app!

I remember watching Minority report again last year and being really surprised watching this scene!! I was like that looks like my research project but this movie is 20 years old???

Spending several years in a VR lab made me start thinking RGBD was some common term, but for sure I'll follow your advice and use more normal words next time. And you were right that it's probably too long since the above title was few letters short the limit already!

By default, mp4 files from iphones do not include depth. The RGBD videos from telegies are custom mkv files that include depth (done by directly using libmatroska, not via ffmpeg), which the webapp can display.

Thanks a lot!! This is definitely a much more interesting video and you have a lovely cat!

Thanks for trying it with the quest and for the feedback. I really haven't thought about this issue. Yes, you are right that I have held the camera with my hand. I'll think about this and probably will need to do something equivalent to video stabilization.

Thank you!! I have no experience in deep learning yet, but I can see it being useful for this system and would like to learn about it!