cj00

OCR on the thumbnail is sometimes good enough for me since I’ll remember the teaser blurb that they put in the image but leave out of the title. So the LLM description service can handle that and maybe it’ll add in “two people standing in front of a boat” and something will hit in a search.

But I’ve actually started pulling in more than just the thumbnail. With the YouTube API I’m pulling in the description and all the tagging and then I use Haiku to do some categorization and auto tagging based on all of that info.

The problem with fetching transcripts is that YouTube likes to soft ban you if you start getting a lot of them at once. At one point I even triggered it just from manually fetching a dozen about ten seconds apart from each other. So now I just leave the transcripts for informational videos where I don’t want to watch the whole thing.

Haha, that would be an interesting LLM fine tuning experiment.

Currently at 3.8 GB for the YouTube thumbnails. An additional 800 MB for Crunchyroll thumbnails.

I used the same concept to build a searchable database of YouTube thumbnails in my watch history. I always end up trying to remember a video I didn't finish from a week or two ago and YouTube's history search plain sucks. So I built an app that tracks all my watch history across all devices, downloads the thumbnails and runs them through a local Gemma model to write a paragraph description and OCR any text in the thumbnail.

Currently at 30,000 thumbnails described, basically 2 years of watch history. Search is via FTS5 in SQLite and works great on an 8 year old MacBook Pro.

I've found some strange guardrails with the Foundation models. Like blocking describing of any photo with a single woman in it. It does this relatively consistently but with a different subset every time. Like out of 10 photos in the set, it's skips 6 then the next run, it skips 3. Have you seen guardrail issues like this?

I switched to Gemma 4 instead.

I built a generative UI app for Apple TV over the weekend. It lets any agent send notifications, questions and media to any of my Apple TVs. Putting it on this device would be perfect, hopefully they let developers start doing third part apps soon.

How well does it handle pronunciation, especially of things like proper nouns? This has been one of the harder parts I've found. On a financial web site I saw one example where the generated audio pronounced initiates as a noun instead of a verb. It's a subtle difference but can significantly change the meaning if worded wrong.

I was trying to use TTS to regenerate some old audiobooks and getting names pronounced correctly is pretty hard unless the model supports something like SSML.

I did something similar and just put the files on an old laptop and used Tailscale. That gives me an https URL and Apple Podcasts downloads the files fine.

This is timely! I'm trying to take control of my Echo Wall Clock which connects to an Alexa device that I want to get rid of. There's very little info on it but Claude was able to find the FCC filings and now we've got lo-res images of the circuit board. It's inspecting the test pads on the circuit board now to see if it can figure out how to replace the firmware.

If they put up a fake camera, what happens when an incident occurs and the footage is requested? Does it just become an internal secret that it’s not a real camera?

On the other hand, installing a real camera means costs associated with monitoring so you also need to convince everyone it’s worth that cost.

For shits and giggles I had Claude build out a full voice cloning pipeline that runs completely locally on my Steam Deck. I’ve got Gemma e4b, Qwen and Chatterbox all running on the AMD.

> 4. Regulatory disadvantages: In everything from company regs, employee regs, unions, privacy regs, data portability regs, etc.

Agreed. My own anecdote: my company is global and for the past 6 months, we've been working on getting regulatory and legal approval for an LLM-based feature. The initial proposals of going live in all of our markets have been pared back to exclude Europe altogether due to the regulatory environment.

When I took part in company-wide gen AI councils that reviewed new product rollouts, it seemed like there was a definite hesitation from higher ups from pushing out any leading edge features to European markets. And it's not that the regulations would necessarily block these features from going live but that they'd increase implementation costs to the point where it wouldn't be worth it.

Yeah it's been years but I remember seeing arguments with Google devs saying if someone had access to your local file system, you're already SOL.

Yeah searching your history is so terrible too I ended up making a custom database that takes the also horrible Takeout output and parses it into a SQLite db. I end up relying on it when I remember some video I started watching weeks ago but can’t remember where it was anymore.