So "Transformers" are part of the attention-based systems, which are a approach for modeling input-output relationships that is an alternative to Recurrent Neural Networks. These are instead based on Convolutional Neural Networks.
The innovation here is that the transformer is compressed, allowing the system to deal with longer sequences.
The innovation here is that the transformer is compressed, allowing the system to deal with longer sequences.