Comment by llwu

This is a cool idea - it could be tested it by seeing whether W_V tends to map <bot> to 0, and whether <bot> tends to get attended to.

There are interesting things to said whether or not this turns out to be the case.