heliographe
Joined 2 years ago
Joined 2 years ago
It's not mentioned in the piece, but for the latter I don't see any technical obstacle to having several of these in different rooms in the house, or a couple in the same room to augment coverage.
One big difference in direction is that Dynamicland + Folk Computer use projector+computer vision as substrate to try and reinvent/rethink the interactive/dynamic medium, and what end-user programming could be (which is very cool).
In that way I am much less ambitious; this exploration is more about how there is an entire class of existing interactions (the ones showed in the piece) that we perform today on screens, that could be much more interesting + collaborative if overlaid in the real world. And I am also terribly uninspired by the eyewear-based computing vision all major companies have pushed lately.
Most of the videos are shot in real life, during summer daylight (only exception are the 2 person around a table ones, which are shot at night in a room with regular ceiling lighting).
And the projector I used is only 1100 ISO lumens; fancier models go way higher.
If you project mostly white content, it's remarkably readable + visible. Outdoors in direct sunlight would be another story, of course.
The main question I tend to get from hacker-types is "what are the tech specs?". The demos run on a Mac with custom software I built for projection mapping/rendering/etc. (the hand tracking uses Apple's built-in framework, which is very good), a small consumer 4k laser projector, and a basic webcam for vision.
But really, this is primarily a research/design prototype - I think it's much more important to figure out what the user experience should ideally feel like, if the interactions are even plausible & desirable in the first place; and then work backwards to the technology.
There's still some amount of denoising/sharpening/etc happening in the ISP, but essentially shooting Bayer RAW gets you the output of just the small first part of the pipeline, rather than the whole pipeline.
[0] Despite the confusingly similar names, Bayer RAW and Apple ProRAW are quite different - I wrote about it on this page because I frequently get emails from confused customers.
I think this is a situation where the enthusiasts care more about the telephoto (for intentional photographic compositions), but the mainstream user base uses the Ultrawide more (for inside shots, group photos, etc) - so Apple caters to the latter, to the dismay of the former.
(the clear wide > tele > ultrawide hierarchy of optical/sensor quality doesn't help either)
> Camera quality feels as good as ever, it’s apparently not as good a main sensor as the Pro iPhones but I can’t really tell. It still has that distinctly over-processed look plaguing modern smartphones
Same here - photography-minded users hate the over processed look, but the general population must like them, otherwise there's no way that manufacturers would persist doubling down on it year-after-year.
For a while now I've been working on a Bayer RAW only pro camera app (https://bayercam.app) specifically because I'm a photography nerd and care about full control, but I realize it's a pretty niche thing.
And as you point out, at capture time you can use color filters to affect the image; processing too can lead to fairly different results based on what developer you use.
This is in contrast to color film, which I find to be much more rigid and narrow in how it’s meant to look and be processed; one could argue there’s much less range for interpretation from negative to final image (especially so with slide film, which completely falls apart if it’s ever so slightly over/under exposed).
I find this to be a key insight. I've been working on a black-and-white film app for a while now (it's on my website in profile if you're curious), and in the early stages I spent time poring over academic papers that claim to build an actual physical model of how silver halide emulsions react to light.
I quickly realized this was a dead end because 1) they were horribly inefficient (it's not uncommon for photographers to have 50-100MP photos these days, and I don't want my emulator to take several minutes to preview/export a full image), and 2) the result didn't even look that good/close to actual film in the end (sometimes to the point where I wondered if the authors actually looked at real film, rather than get lost into their own physical/mathematical model of how film "should behave").
Forgetting the physics for a moment, and focusing instead on what things look and feel like, and how that can be closely approximated with real time computer graphics approach, yielded far better results.
Of course the physics can sometimes shed some light on why something is missing from your results, and give you vocabulary for the mechanics of it, but that doesn't mean you should try to emulate it accurately.
I read this interview with spktra/Josh Fagin and how he worked on digitally recreating how light scatters through animation cels, which creates a certain effect that is missing from digital animation - and it was validating to read a similar insight:
"The key isn’t simulating the science perfectly, but training your eye to recognize the character of analog light through film, so you can recreate the feeling of it."
https://mastodon.social/@heliographe_studio/1156653713048409...
(taken with BayerCam.app, not Halide, but Halide can capture the same raw Bayer data)
It's not an amazing photo by any means. But it is a photograph of the moon - the seas are all well delineated, Copernicus/Kepler/Aristarchus/Grimaldi are visible/recognizable.
A test that smartphones did not pass a few years ago.
I see it a lot in photography, to show before/after processing - but what you want to be able to quickly compare are the same part of an image with and without the processing applied.
One of the photography tools I make is a LUT viewer/converter - and while I didn't have the slider at first, I guess it's standard enough at this point that people asked for it and I added it.
But I made two additions to it that make it more useful IMO:
- have labels on the left/right top corners, so it's immediately clear which version of the image you're looking at
- click and hold on the image to preview the full unprocessed version; release to revert to the view. That makes it easy to quickly compare the two versions of the same spot of a photo. (similar to what you suggest, but non-latching)
I have a video of it in action here:
I’m an independent developer based in Tokyo, building tools for photographers and visual artists.
Before that, 13 years in the Bay Area in startups/big tech.