a1k0n

Yeah, it is exact in this specific circumstance. But yes, it's exactly the same trick; I also enjoyed that video in my Youtube recommender feed last week!

lmao! that was so cringe I unlisted it from the main index. who is "we"?

part 3 is to find the secp256k1 private key for satoshi's bitcoins

Yes, it has to in fact. If you have zero context to attend to in a transformer, and you try to predict the first token, you effectively are multiplying a zero-vector by the attention head, making all tokens equally likely in the final softmax (unless the lm_head has a bias, but at least in GPT it does not).

So the <|beginning of text|> token, with no context before it, learns to predict the first-token-in-a-document distribution. That's not quite the same as predicting nothing at all.

For finding interesting Ethereum addresses on the GPU (uses SHA3 instead of MD5) there's a tool called profanity.

Amusingly the tool was archived by the author years ago, and just last year a stupid bug in the random initialization code was found (32 bytes of state initialized with ~31 bits of entropy), leading to ~millions of $ in stolen ETH by exploiters.

> If anyone has an idea why the TIA’s designers used LFSRs for this stuff, I’d love to hear about it.

I'm guessing because a counter would have ripple carry (i.e. five extra gate delays when rolling over from 111111 to 000000) or need extra gates for carry lookahead and an LFSR is constant-delay.