Comment by gwern

One fun part - we used the inline metadata trick to train a single GPT-2-1.5b to do all the different subreddits. It allows mutual transfer learning and saves an enormous amount of space & complexity compared to training separate models, and it's easy to add in any new subreddits one might want (just define a new keyword prefix and train some more). Not sure that trick is meaningful for Markov chains at all!