mangoman

Joined 15 years ago

Karma
1,609
Submissions
139
Member since
Jan 18
https://prashanth.world

meet.hn/city/us-Durham

Socials: - linkedin.com/in/prashanth-sadasivan - github.com/prashanthsadasivan

---

View on Hacker News

Recent submissions

Comment · 2 months ago
I don't think it's the same as management. As a manager, you're not responsible for the code your team creates. You're responsible for their output, but that's fundamentally different. I also think you're managing people's emotions, expectations, holding people accountable, etc. Maybe those who enjoy working with agents would actually rather be PMs?

I agree the shift feels different than e.g. going from jquery to react or something.

Comment · 2 months ago
Based on the SynthID-Text paper https://www.nature.com/articles/s41586-024-08025-4 I agree that the LLM's learned distribution isn't modified, but I don't think it's correct to say that the sampling process is not modified. Also I just read the paper today so I could be misinterpreting things.

As described in the paper, you're right that it doesn't affect the main sampling technique, but what they do is they sample the distribution for 2^m samples, and then use Tournament sampling to choose the tokens among those 2^m samples, and the watermark key changes the scoring of the tournament options, using the watermark key as an input to the random generator that generates the scoring functions.

Then, to calculate the watermark, they take the text, and compute the mean g-values of the text, and a higher score means that it's more likely that it was sampled using the provided selection of tournament watermarking functions.

let's say you had some top P words: mango, banana, pineapple, guava, and you sampled 8 times, and got each one twice in the following order:

1. mango 2. banana 3. pineapple 4. guava 5. mango 6. banana 7. pineapple 8. guava

without tournament sampling, you'd truly see any of those come through. But in tournament sampling, you take those 8 options, create m scoring functions based on the pseudorandom generator, and score the 'tournament' by sampling the biased distribution you create from the g values. That does change the sampling from based purely on the LLM and entropy, but i mean, if the watermark key is also generated from some entropy, it's probably representative of the original sampling options as expected?

this is a very fascinating topic! I do still stand by my point that anthropic is the only one who can tell if something is watermarked or not and feeling icky, but the paper has mostly quelled my concern on impacting the intelligence part.

Comment · 2 months ago
huh, thanks for commenting this! I trusted the linked explanation post https://declaude.org/watermarking/ but actually reading the the synthid paper showed me that my understanding was wrong: https://www.nature.com/articles/s41586-024-08025-4

It is definitely blurrier whether you can say this approach changes the distribution then. By definition, it _has_ to change the probabilities of output tokens, but it's not totally clear that the pseudorandomly generated scoring functions does affect the learned distribution.

put another way, I think it's safer to do:

compute distribution -> sample -> watermark from sampled options

than it would be to do:

compute distribution -> watermark distribution -> sample

Comment · 2 months ago
I agree, but thats also learned, and it is done to encourage specific responses for a task, different than applying a mask to the distribution based on a key.
Comment · 2 months ago
10% is a lot! And why would this slow down the cheating epidemic? There’s tons of ways like declaude etc to get around the check. Also Claude’s style is very distinctive (e.g. “load bearing”) AND has changed since 4.5 quite dramatically. I might not be able to tell on a specific response, but I can tell you that I went from canceling my ChatGPT plan in November, to now reaching for it first and considering cancelling Claude because its style is getting really groan-inducing. It’s weird because I thought ChatGPT was really annoying about a year ago, and now codex is my first choice.

Is the cheating epidemic so bad? I’m a little out of the loop there truthfully, what are the consequences of not being able to detect AI generated text in non academic settings? And in academic settings… maybe I am underestimating the challenge, but it does feel like the assignment and ways education happens needs to change?

As an aside, I’m not totally sure why this solution feels so icky to me. There’s something Orwellian about how the phrasing of a passage embeds hidden information that only Anthropic can see i guess

Comment · 2 months ago
I’m surprised by the comments here being so favorable to anthropic. The comments are right about there being no “best” token, and yeah Gruber may have an agenda here. But I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model. Take the “gray” and “overcast” choices. And lets say before applying synthid the percentages were 48% and 52%. Those percentages were learned from the training data and RL. To change those percentages to 45% and 55% in a non learned way makes it seem like training wasn’t important? Or more likely they dont have the data that shows the failure modes? also, don’t these choices compound the changes to the distribution in later sampling choices?

It is a bit of a mystery to say that “its okay to choose different tokens that we would have for watermarking bc people don’t notice” as though word choice doesn’t matter. If it doesn’t matter, doesn’t that mean that intelligence is more of a commodity than they would want it to be?

Comment · 2 months ago
What are you talking about? I understand OP has the perspective of writers, but let’s say you’re asking an LLM to explain a concept and it uses green words that are actually more difficult to understand. Or you ask for an analogy to explain a topic and the analogy doesn’t quite land because the description used “gray” instead of “overcast”.
Comment · 2 months ago
I’ve been trying to start this “I don’t need any distraction for this task” challenge, but honestly it’s been REALLY tough for me. I’ve even noticed apprehension to start a particular chore like folding laundry if I don’t have something to distract me. Its like… because folding laundry takes so long, without a distraction it feels like I’m wasting my time, or just running out of time in the day?

Doing dishes, mowing the lawn, even just waiting for agents to finish, this need for distraction has really taken a hold of me, and it’s even getting in the way of me wanting to do things that I feel would motivate me again.

I can’t help but wonder if agentic development has exacerbated this problem for me. There are so many more times where I just have to wait for things now. Sometimes I can multitask, and work on two things at once, but often that ends with me doing both things with less craft. So I end up consuming content more, which feels addictive too… and I’m not even talking about TikTok or Instagram or Youtube Shorts (and i don’t judge those who do consume there). I mainly consume longer form Youtube, Podcasts or honestly, TV shows that I’ve seen multiple times before.

It’s like the agentic tools incentivize NOT thinking very hard, and so I do it less, making it more intimidating to start thinking hard about a problem because it feels like a waste of time. Which, paradoxically, is why I think I am consuming much more content than I used to, filling up space with intellectual junk food.

Comment · 2 months ago
> The world does not wait with bated breath for product launches. A marketing campaign will not change the world. Almost no one will remember all the work we do.

It did at one time! I think that’s what’s so alluring initially about Workism. People waited with bated breath for the iPhone and iPad and its new versions. Those iPod commercials and the marketing campaigns for “rip. Mix. Burn.” did change the music world.

But the quote is also right. It doesn’t happen anymore, and I think that’s why the religion of Workism is especially vulnerable at this point.

In someways, it sorta feels like the article is cheering AI on? Like, it judges Workism, and goes on to say that AI shatters the illusion, but then.. that’s it? Deriving value from good work is pointless now and we should all just go volunteer?

I dunno, i clearly identify with those who loved work and now feel a bit lost, and yes i have been thinking about alternatives, but I feel like there must be knowledge work that is still valuable both financially and socially and also provides some sense of pride and purpose… if not, i fear the situation is much more dire than we are prepared for

Comment · 5 months ago

  This release unifies those capabilities with a Mixture-of-Transformers (MoT) architecture built around two towers. 
  Reasoner tower: A vision-language model (VLM) ... This serves as the ‘brain’ that reasons about the world before any generation happens.
  Generator tower: Generates future observations and action sequences. This tower uses a diffusion-based process to generate physics-aware video and action outputs that are conditioned on the reasoner tower’s understanding.
This sort of approach (and others i've seen like it) always appeal to my inner engineer, trying to optimize and balance tradeoffs between model architectures and combine two things to yield the best of both worlds

But based on my understanding of the Bitter Lesson (http://www.incompleteideas.net/IncIdeas/BitterLesson.html), this is precisely the wrong approach in the long term. I'm linking the actual text of the bitter lesson because I think it's misunderstood (or I just don't agree with how i've seen it used in discourse). Specifically:

  The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach. 
This architecture feels specifically like "trying to build knowlege into the agent that will help in the short term" but will plateau long term. That's not to say that there won't be some interesting learnings or things built on top of it, but I doubt that there's a lot of juice to squeeze with this kind of approach IMO.
Comment · 7 months ago
I dunno, I thought that too for a while too, but there are a lot of new ideas in terms of architecture that may warrant massive training runs. Mamba and state space models are pretty interesting, but haven’t had their transformer moment yet because I haven’t really seen anyone go for broke on training it with a huge data set and model size. Even some of the more fundamental changes too like Kolmogorov–Arnold Networks or some of the ideas behind continuous back propagation haven’t really had the opportunity to be pushed to the limit. I think it’s still early days on what these models can do. And I say this as someone who bought a Mac m3 max 128gb ram, based on the hope that the on device training and inference work would eventually move locally. It’s encouraging to see the progress though and I hope it does move locally though.
Comment · 8 months ago
There’s something off-putting about making a blog post about some splashy tech that’s is a fork of an open source project, and that tech not also being open source? It reads to me like “Hey, we thought the open source goose project was just okay, so we forked it to do it better. But we’re not going to contribute it back to and instead rename it.”

I think it probably wouldn’t be as weird if the project were a meaningfully different fork of it, but it sounds like it’s trying to accomplish the same goals as the open source project which I feel should probably be ported back? and renaming it seems sorta ungrateful? Kinda like that “you made this? I made this” meme. Maybe I just don’t have an understanding of how different the projects are though…

Comment · 8 months ago
I guess what’s wrong with it? Let’s say it has read only access, new messages and calendar invites need approval. I’m not sure I understand the harm? I suppose data exfiltration, but like you could start with an allowlist approach. So the first few uses and reads take a while with allowing the ai to read stuff , but it doesn’t seem that crazy given it’s what we basically do with ai coding tools?

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton