noddybear

This stuff (activation steering) has been around for a few years at this point

I know one of these researchers personally (Joe Benton) and he's not the self-promoting type. I believe his concerns are genuine.

I don’t think the runs need to be aborted… you can just release a mid-training checkpoint!

It’s an interesting contrast that you claim that the USA does not equate to its current political situation, whereas conversely the EU is ‘rotting’ from the inside. Do you think this is a balanced and reasonable claim?

Maybe. I would contend that wealth accumulation over the long term is a function of the strength of local institutions e.g educational, judiciary etc. This is effectively the thesis of ‘Why nations fail’. Europes relative decline will only continue if the institutions of the US (and others) continue to be robust and healthy. I am not sure if this can be relied on into the future.

Right, the purpose of a system is what it does.

Aren’t Unicode characters generally treated as 2 tokens to avoid a huge vocabulary?

Nobody human considers a corporation to be a person. It is merely a legal expedient.

He also believed in the immortal soul that survives after death, so he may have had a more literal meaning.

Most of the functions that LLMs perform aren’t bijective, though.

What prompt constructs the output ‘The answer is 3’ or ‘Yes that’s a great idea’?

Thinking ‘God I wish these people would die’ could increase its propensity to kill all people, even if that propensity is still vanishingly small almost all of the time.

A lot of people are walking around with crazy thoughts. Some of them harm.

An EMP from a high altitude nuclear detonation would do the trick.

Nah, its all pattern matching. This is how automated theorem provers like Isabelle are built, applying operations to lemmas/expressions to reach proofs.

I am really keen on plugging into Age of Empires 2 - although practically I think we need a couple of years of improvements before LLMs would be smart/fast enough to react to the game in realtime. Currently they can't react fast enough - although specially trained networks could be viable.

Thank you!