Comment by fastball

The problem with the rest of inference is that changes are not trivially correct or incorrect, as they are with the tokenization layer.