yusefnapora

It was marketing for their only real product: Tesla stock.

I mean... this has been the most common response to pretty much all LLM criticism since GPT 3? I have yet to see any substantial critique of LLMs that isn't immediately met with "but have you tried the latest models? they're so much better now!"

The article is also clear that it doesn't really matter whether the thing "works" or not - the process of using it is no fun for the author, so why would they want to use it for things that are supposed to be fun?

A rule doesn't have to be perfectly enforceable in order to serve a useful purpose. The rule assumes good intent on the participants, and it communicates what kind of behavior is welcome vs unwelcome. The submitter knows whether they used an LLM or not. If they want to lie about it, that's on them - I'm sure they can come up with whatever justification they want for that, but it's ultimately antisocial and gross behavior.

It's hard to interact with a particular dialect day in and out without it eventually seeping into your habits of speech and thought. So the people most likely to care are your friends and family once they get bored of hearing about the "key insights" you've been having all day.

So confusing how holding people in confinement, starving them, and selling their labor to others without compensation gets people tossing the word "slavery" around. Obviously it's the people paying wages to voluntary workers without giving the government a cut that are the real slavers!

It's not that nobody likes it, in fact the problem is that people like each instance of it well enough in isolation. Millions of people think it's "good enough," so it gets amplified and repeated until every PR description starts to sound like a toothpaste jingle.

The proposed bill would still allow you to choose to be on-call. Your boss just has to compensate you for your time explicitly when it falls outside of agreed hours, instead of getting an implicit claim to your entire life by virtue of your employment agreement.

Presumably Suzie also serves the people in her community who want tanned skin but don't like the sun. A data center offers nothing of value to the community it's embedded in - neither jobs nor any useful product or service. All the benefits accrue to the owners, and the very real costs are paid by the people breathing in the fumes from the gas turbines, or paying an extra hundred bucks a month for power.

If you don't understand why people don't want them, you're probably trying not to. It's not that hard.

No one is "making things insecure" here - it was already hopelessly insecure by design. The author is just revealing the uncomfortable truth.

> ensure this doesn't happen again with more deliberate and effective prompt injection

How can this possibly be accomplished? Even if every actor in the open source world with good to neutral intentions decides that this is anti-social behavior, that does absolutely nothing to secure your system against people with bad intentions.

A system that pulls in arbitrary unstructured text input and treats it as trusted instructions is insecure by design. Asking the entire world to sanitize your inputs for you is a choice - good luck with that.

The project README has explicitly asked users not to use LLMs for years before adding this "malware" to the output. Since LLM users seem incapable of understanding consent, apparently a more firm reminder was needed.

As mentioned in the blog post, if your system is susceptible to this kind of "attack," what is your plan when someone with actual malicious intent gets involved?

You consider it a malfunction for your system to not accept and execute untrusted inputs? And now it's the responsibility of _every program that produces text output_ to tailor the output so as not to cause you problems?

I feel like I'm taking crazy pills here. Time to log off for a while, I guess.

Unfortunately, it seems that few of the people willing to pay money for software development actually care whether you understand the code or not. Unless it breaks in a publicly embarrassing way, of course. Then "you own the code" and the sacrificial human in the loop can be dredged out to take the blame.

All the incentives nudge you towards less and less critical evaluation of the output. The results of careful evaluation are much harder to measure than "this guy is cranking out 10x the code compared to last year!" And while you were busy thoughtfully internalizing the output, the guy next to you has been letting Strega Nona's pasta pot go brrr and spew another thousand lines of spaghetti on top, ready for you to review. Eventually "lgtm" becomes the default, and "do you want me to go ahead and implement that for you" starts sounding like the only way to keep your head above water.