Comment by chronolitus

and here's a breakdown of the architecture:

http://dugas.ch/artificial_curiosity/GPT_architecture.html