yunusabd

I should mention that pasting the code currently only works in the Android app, with the iOS version being stuck in review (again).

I think parent comment was surprised that there was _any_ moving of walls after the fact, which "reducing" implies..

Just out of curiosity, why would the LLM need network access for this? I.e. feeding the doc to an LLM and asking "is this sensitive information according to these criteria: [...]" should get you there most of the way, no? Probably need a handful of (carefully designed) tool calls and a human in the loop somewhere, but it seems achievable.

It sounded like there would be a big value unlock. Depends on your circumstances of course.

Create an anonymized/obfuscated copy of your data and let the agents use that?

I had similar thoughts. The readme intro explicitly mentions hallucinations, that's why I thought I'd ask.

If you're dealing with uid in -> uid out, where you're hoping to get the same uid out, intuitively the entropy would be greatly reduced anyways. Then the question becomes, are words conducive to keeping input->output consistent, given the way LLMs work (e.g. attention mechanism)? I could see it go either way, that's why I'm supporting the idea of running your experiment.

Okay, but you can also validate uids. What I'm asking is whether the human readable uids cause fewer hallucinations, as that would be the real win imo.

That's nice, I've had the issue where LLMs would return non-existent uids. But does this package actually help with that? Token savings are nice, but not really my main concern. If this can measurably reduce hallucinations, it would be really useful.

> Where UUIDs cost ~23 tokens and get hallucinated by LLMs, id-agent produces memorable word-based IDs at ~14 tokens with equivalent collision resistance.

Didn't expect it to get hammered like that, just added caching for the sheets request. Thanks, my guy ;)

Backfilling it further is definitely in the cards, I just want to stabilize the methodology first.

If a comment just mentions Opus without being more specific and in the absence of relevant context clues, it gets mapped to Opus Latest. So it's saying more about the model family than a specific version. Tbh I'll probably remove all "-latest" data points going forward, as I mentioned in another comment.

Yes! Going forward I'm definitely doing that, once there is enough data. Might even backfill the data more into the past. I just want to stabilize the methodology before burning more tokens.

And it's probably a good idea to create a list of model release dates, so older comments can't accidentally map to models that weren't released yet.

From the comments that I've checked manually it's pretty good. You can go to the "User Ratings" tab in the Google Sheet and check some comments to get an idea.