marmaduke
Joined 14 years ago
Karma
2,806
Submissions
1,519
Member since
Apr 10
[ my public key: https://keybase.io/wdmn; my proof: https://keybase.io/wdmn/sigs/KaH_BqIpR_cc-JNsLgWnzWCnKy_ZnbQfoKTC0ms3-1o ]
View on Hacker NewsJoined 14 years ago
and i tried to unpack it a bit here https://wdmn.fr/rank-1-take-on-rwkv7s-in-context-learning/
jax is fun but as effective as i’d like for CPU
covers some of the recent perspectives on this modeling approach if you’re interested.
perhaps also more precisely they also have quite an advantage on anything that needs and plays nicely with caches? when I sliced my problem to maximize cache usage, I saw pretty clear scalability with cores: L1/L2 cache bandwidth is ~30GB/s, so e.g. a 32 core system starts to compete with the big consumer GPUs.