mangoman
Joined 15 years ago
Joined 15 years ago
I agree the shift feels different than e.g. going from jquery to react or something.
As described in the paper, you're right that it doesn't affect the main sampling technique, but what they do is they sample the distribution for 2^m samples, and then use Tournament sampling to choose the tokens among those 2^m samples, and the watermark key changes the scoring of the tournament options, using the watermark key as an input to the random generator that generates the scoring functions.
Then, to calculate the watermark, they take the text, and compute the mean g-values of the text, and a higher score means that it's more likely that it was sampled using the provided selection of tournament watermarking functions.
let's say you had some top P words: mango, banana, pineapple, guava, and you sampled 8 times, and got each one twice in the following order:
1. mango 2. banana 3. pineapple 4. guava 5. mango 6. banana 7. pineapple 8. guava
without tournament sampling, you'd truly see any of those come through. But in tournament sampling, you take those 8 options, create m scoring functions based on the pseudorandom generator, and score the 'tournament' by sampling the biased distribution you create from the g values. That does change the sampling from based purely on the LLM and entropy, but i mean, if the watermark key is also generated from some entropy, it's probably representative of the original sampling options as expected?
this is a very fascinating topic! I do still stand by my point that anthropic is the only one who can tell if something is watermarked or not and feeling icky, but the paper has mostly quelled my concern on impacting the intelligence part.
It is definitely blurrier whether you can say this approach changes the distribution then. By definition, it _has_ to change the probabilities of output tokens, but it's not totally clear that the pseudorandomly generated scoring functions does affect the learned distribution.
put another way, I think it's safer to do:
compute distribution -> sample -> watermark from sampled options
than it would be to do:
compute distribution -> watermark distribution -> sample
Is the cheating epidemic so bad? I’m a little out of the loop there truthfully, what are the consequences of not being able to detect AI generated text in non academic settings? And in academic settings… maybe I am underestimating the challenge, but it does feel like the assignment and ways education happens needs to change?
As an aside, I’m not totally sure why this solution feels so icky to me. There’s something Orwellian about how the phrasing of a passage embeds hidden information that only Anthropic can see i guess
It is a bit of a mystery to say that “its okay to choose different tokens that we would have for watermarking bc people don’t notice” as though word choice doesn’t matter. If it doesn’t matter, doesn’t that mean that intelligence is more of a commodity than they would want it to be?
Doing dishes, mowing the lawn, even just waiting for agents to finish, this need for distraction has really taken a hold of me, and it’s even getting in the way of me wanting to do things that I feel would motivate me again.
I can’t help but wonder if agentic development has exacerbated this problem for me. There are so many more times where I just have to wait for things now. Sometimes I can multitask, and work on two things at once, but often that ends with me doing both things with less craft. So I end up consuming content more, which feels addictive too… and I’m not even talking about TikTok or Instagram or Youtube Shorts (and i don’t judge those who do consume there). I mainly consume longer form Youtube, Podcasts or honestly, TV shows that I’ve seen multiple times before.
It’s like the agentic tools incentivize NOT thinking very hard, and so I do it less, making it more intimidating to start thinking hard about a problem because it feels like a waste of time. Which, paradoxically, is why I think I am consuming much more content than I used to, filling up space with intellectual junk food.
It did at one time! I think that’s what’s so alluring initially about Workism. People waited with bated breath for the iPhone and iPad and its new versions. Those iPod commercials and the marketing campaigns for “rip. Mix. Burn.” did change the music world.
But the quote is also right. It doesn’t happen anymore, and I think that’s why the religion of Workism is especially vulnerable at this point.
In someways, it sorta feels like the article is cheering AI on? Like, it judges Workism, and goes on to say that AI shatters the illusion, but then.. that’s it? Deriving value from good work is pointless now and we should all just go volunteer?
I dunno, i clearly identify with those who loved work and now feel a bit lost, and yes i have been thinking about alternatives, but I feel like there must be knowledge work that is still valuable both financially and socially and also provides some sense of pride and purpose… if not, i fear the situation is much more dire than we are prepared for
This release unifies those capabilities with a Mixture-of-Transformers (MoT) architecture built around two towers.
Reasoner tower: A vision-language model (VLM) ... This serves as the ‘brain’ that reasons about the world before any generation happens.
Generator tower: Generates future observations and action sequences. This tower uses a diffusion-based process to generate physics-aware video and action outputs that are conditioned on the reasoner tower’s understanding.
This sort of approach (and others i've seen like it) always appeal to my inner engineer, trying to optimize and balance tradeoffs between model architectures and combine two things to yield the best of both worldsBut based on my understanding of the Bitter Lesson (http://www.incompleteideas.net/IncIdeas/BitterLesson.html), this is precisely the wrong approach in the long term. I'm linking the actual text of the bitter lesson because I think it's misunderstood (or I just don't agree with how i've seen it used in discourse). Specifically:
The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
This architecture feels specifically like "trying to build knowlege into the agent that will help in the short term" but will plateau long term. That's not to say that there won't be some interesting learnings or things built on top of it, but I doubt that there's a lot of juice to squeeze with this kind of approach IMO.I think it probably wouldn’t be as weird if the project were a meaningfully different fork of it, but it sounds like it’s trying to accomplish the same goals as the open source project which I feel should probably be ported back? and renaming it seems sorta ungrateful? Kinda like that “you made this? I made this” meme. Maybe I just don’t have an understanding of how different the projects are though…
meet.hn/city/us-Durham
Socials: - linkedin.com/in/prashanth-sadasivan - github.com/prashanthsadasivan
---