k1m
Joined 16 years ago
Karma
6,926
Submissions
601
Member since
Oct 14
I'm a software developer interested in web development, digital publishing systems, media and politics. https://www.keyvan.net
View on Hacker NewsRecent submissions
Comment · 5 hours ago
That's really interesting! Thanks for sharing. Will be trying it soon.
Comment · 10 hours ago
> Having less diversity in answers could mean you're more consistently getting a better/worse/same quality answer.
I wrote the article. I'm trying to show how diversity loss is itself a measure of quality loss in many applications. And as I pointed out in another comment, the paper's authors themselves acknowledge as much in their paper.
> Instead of just re-rolling the answer, why not continue in that thread and say what you don't like?
The re-roll is actually a feature of many chat apps with a dedicated button. So it's something users are familiar with.
But the results are also quite different. The prior answers in your example of asking for x alternatives influence the following answers too much in a way a re-roll doesn't.
Comment · 11 hours ago
I wrote the article. How you define quality is exactly the issue.
> In my opinion, 'sample diversity' and 'sample quality' are not the same thing, and conflating the two concepts leads to poor intuition.
But you're assuming everyone has the same notion of quality as you. As I point out in the piece, even the paper's writers use "quality" to show how watermarking can "lower the overall quality of a system"
> SynthID and the like probably matter most if one is trying to repeatedly sample an LLM in order to infer the probability distribution
Which is another impact I write about in the piece. If you evaluate the output quality of an LLM system in this way, you're impacted more by watermarking.
Comment · 26 days ago
Worth noting that this is what OpenAI wrote about text watermarking two years ago:
> While it has been highly accurate and even effective against localized tampering, such as paraphrasing, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character - making it trivial to circumvention by bad actors.
> Another important risk we are weighing is that our research suggests the text watermarking method has the potential to disproportionately impact some groups. For example, it could stigmatize use of AI as a useful writing tool for non-native English speakers.
https://openai.com/index/understanding-the-source-of-what-we...
Comment · 2 months ago
It reduces diversity, which they don't talk about much. Wrote about it here. https://blog.keyvan.net/p/ai-text-watermarking-and-quality
Comment · 2 months ago
Agree. I think many people forget that not long ago, HTML markup on many sites was a lot richer than it is today. Making it trivial to produce a good trimmed down markdown version.
The reason it may be more difficult today is because we've lost a lot of that. Some of it because of modern JS frameworks, but some also because publishers simply don't want to make it easy for the useful stuff to be scraped and extracted easily.
I'm not convinced that's changing because of AI agents (it's getting worse in many ways with anti-agent rules). Maybe improving for documentation pages intended for agents. But if it is changing, I think it'd be far easier to improve the HTML and let the agent take care of the rest.