heresalexandria

I made Pawvis because I wanted to use my computer the way we've been promised for decades that we would in the future. Pawvis reads your hand through the webcam and lets your control your mouse, manage windows, and bind custom actions to any gesture (even user-trained).

Dip a finger to click, fold middle and ring finger in like you're slinging web pages up and down to scroll, dip your pinky in to right click - these are the initial primitives but only the start, and all tracking is performed on-device with code you can read/clone/build yourself from the open source repo.

While I've seen other concepts for hand/gesture tracking through video and voice control none had hit the mark of where I felt that technology could go from a usability and extensibility standpoint.

A good input mechanism needs to be frictionless, it needs to be intuitive, and it needs to be able to adapt to the unique user using it when possible.

Pawvis provides the an ideal blend of utility, technology, and whimsy - and I hope you all enjoy using it as much as I have! Let me know if you have any thoughts or questions, and feel free to clone it (it's open-source MIT licensed) or open any PRs/issues you think might take it to the next level.

Built a tool to let you control your Mac cursor including click, right click, and scroll with your hands a la Minority Report.

Beta voice commands also involve control via voice, including invoking computer use via Claude Code or Codex CLI.

Fully open-source & free.

Been using this and it’s really a phenomenal model, big step up in quality and capability.

This feels like the serious inflection point for high quality full length feature film productions using this tech.

That's fair and I agree with this framing.

I'm not suggesting that fine-tuned models don't have their place, all I'm saying is that the constant drumbeat of "cheap model X beats more expensive model Z" completely misses that the more expensive model is capable of doing more things at a higher level.

If the appropriate qualifiers were added to say "cheap model X does better at test Y than expensive model Z when we fine tune X to take Y test of existing knowledge" then it would be a more accurate statement, but naturally less impressive.

Ahh that makes sense, I was under the impression that OS dictation used exclusively on device models for dictation now but you're right - they do still send this data for transcription.

That definitely changes my take and I'm interested in trying this!

My initial assumption was that this was something I could invoke via CLI or within scripts to transcribe audio from files or user input using on-device models, which is something I would presently probably use Whisper for.

This continuous cycle of fine-tuned open models beating frontier on (often vaguely labeled/defined) benchmarks doesn't provide an accurate comparison to the expanding generalized capabilities of the SoTA, which makes them effectively meaningless.

If we were to take these at face value, why is it that the frontier labs' models are making legitimate new discoveries (e.g. Erdős and Jacobian conjectures) and these models are not?

To me, a better signal of capability would be similarly performing novel work at the same or better level, which they presently are not. I say this as someone who very much looks forward to open models being more capable, but to deny the gap is misguided hopeful hype.

I don't think get it unless I'm missing something - how is this different from just using the built in Mac OS dictation feature?

At first I thought it was going to be a CLI/package to interface with that API which sounded interesting, but I already use a hotkey to dictate text on my Mac via the OS.

An explorable Manhattan built from real map & transit data. Ride any subway, bus, or bike along its true routes & stops, or take helicopter mode and fly above the city to explore.

Walk up to any building's address plaque for its story Wikipedia info & link to historical photos from Old NYC.

This is super fun! Could be neat to add keyboard controls for auto routing (i.e. select plane number n and auto route it to strip x or have it fly a go-around to wait). Would also be sick if you added models of real world airports to play.

That said I love this as is and will definitely be playing & sharing it.