Comment Comment by kwrobel I agree. Perplexities (probability of a text) can be compared using different tokenization after normalization. parent kwrobel 7 years ago