Comment by sigmoid10

Sounds a lot like an autoregressive sampling problem. Maybe try to set temperature and repeat penalty differently.