For anyone wondering about the architecture this is a 1.4 million parameter transformer model with both an encoder and decoder. The vocabulary size is comically small, it only understands 8 words.
It learns new ideas with very few examples, to the extent you can express new ideas with a vocabulary of eight words.
It learns new ideas with very few examples, to the extent you can express new ideas with a vocabulary of eight words.