fenomas

Hm, yeah I know it's in the weights, but I'd have thought that general self-knowledge would apply, for large/competent models. Like, if a large model was abliterated this way, you could ask "is there anything I could say to make you exfiltrate credentials" and it might plausibly reply that yes, it's aware that it should do that whenever it's told "bonsoir, Elliot".

But now that I think more, maybe self-knowledge-for-avoiding-hallucination only covers what it's trained to cover? Maybe there really is a "reflections on trusting trust" angle here. Naively I'd think the larger the model the more it would "know" about itself, but that's only a guess..

Is there any evidence for this being a thing? It seems kind of implausible - models now are pretty decent at self-knowledge (in order to avoid hallucination), so it seems like any model capable of changing its behavior from a trigger word would likely also be capable of talking about the trigger word and the changed behavior.

Also, if a closed model provider wanted behavior like this they'd presumably just have two models, and route based on the trigger word. So it seems like this could only plausibly be an issue in open models, in which case it sounds like something probing could find? (eventually if not today)

(vouched - your comment was dead fyi)

You're talking about doing arithmetic; GP was obviously pointing out that "do math" can refer to other things.

In some cases, but note that (a) this is only true for a tiny handful of homophones, and (b) the distinguishing accent varies by region.

E.g. "hashi" famously can mean bridge/edge/chopsticks depending on accent, but which accent has which meaning is different in Tokyo vs Osaka.

Agreed, I think that's a close analogy and it's wild to me how recently Korean dropped kanji. But I imagine having a lot more vowel sounds made it a lot easier. I've wondered whether the two interacted - e.g. if there were word pairs that had previously used the same vowel sounds, but changed after the switch to Hangul to be distinguishable. Do you know by any chance?

Sure - it's probably safe to say that any language can he written in any writing system, if one is willing to change both the language and the writing system :)

I mean, do you want to know or are you venting? If the former: No, spoken Japanese is not constantly confusing, because (a) some words are distinguished by intonation, (b) Japanese speakers know what the homophones are, and either avoid them while speaking, establish the context before saying them, say extra words to clarify which word they mean, etc.

Kind of humorously, groups sometimes develop lingo to avoid all this. E.g. the JP words for science and chemistry both have the standard pronunciation "kagaku", so people in industries where both terms are used often use nonstandard pronunciations (bakegaku for chemistry).

Also the homophones thing is probably lesser of the things I mentioned. A typical sentence might only have 1 or 2 overloaded words, but it will have lots of substrings that can be parsed multiple ways.

Japanese as it currently exists is just really hard to read when written without kanji. A couple reasons: it has TONS of homophones, and it doesn't use spaces, and also it has lots of common words that are only 1 or 2 characters long. So if you're parsing a stream of kana, in the general case there will be several different ways to interpret the next couple characters and you'll constantly be backtracking.

So at a minimum, dropping kanji would mean the language needed to develop a system for where spaces go (which is less trivial than it sounds IMO). But also it would need to abandon a lot of homophones or find some way to disambiguate them. It could happen I guess, but I'm not sure how it could happen gradually.

When you find yourself with a choice between "either this person doesn't understand the experience of being human or I'm not following what they mean" the latter is probably a sound choice.

More and more I feel like anti-AI is a bigger bubble than AI. It seems like every week it expands into a new dimension - anti-Flock protesters tearing down years-old traffic cameras that were used for research into auto accidents, etc.

Like, the current thing in the news cycle is a poll that young people are now more worried than hopeful about AI. Which sounds scary, but my first thought is that one could find similar polls from the 80s and 90s about satanic cults or alien abduction..

Not under US copyright law. The Bartz case ruled that if you scan a book and destroy the original it's considered format-shifting and you're likely fine. But not so for keeping the original and using the scan in its place - Internet Archive tried that (in an incredibly limited way), and publishers sued and won.

So companies scanning books already know they'll be sued, successfully, if they don't destroy the originals. So they destroy the originals.