Comment by doubtfuluser

But why using an encoder model instead of a BERT based model? For a pure classification that should be easier to train and work quite well