Trained from scratch and running on the Game Boy Color.
Rei has memory registers, emotional state and 192 bytes of persistent “soul state”. Leave her alone and she gets bored and starts observing the world :)
~2 tok/s on actual hardware.
The challenge of doing LMs without multiplications or divisions in HW and getting it to interactive speeds.
Rei has memory registers, emotional state and 192 bytes of persistent “soul state”. Leave her alone and she gets bored and starts observing the world :) ~2 tok/s on actual hardware.
The challenge of doing LMs without multiplications or divisions in HW and getting it to interactive speeds.
ROM, source and training code in repo