Comment by bobbyprograms

Seems like hallucination will always be an issue for predict the next word training. Maybe we need to rethink pretraining .