BTW - Now that discussions about sampling techniques within NLP are going mainstream, I have to shamelessly self-plug my own research work in this direction, specifically in the context of Constrained Text Generation. Simply filtering tokens according to constraints before generating text is wonderful for fixing stupid failure cases in LLMs like ChatGPT - such as asking it to not write with a particular letter, asking it to respect syllable counts, or asking it to rhyme properly (look carefully, ChatGPT struggles with exact rhymes in poetry!)
https://paperswithcode.com/paper/most-language-models-can-be...
My work got cited by Gwern in this article! https://gwern.net/gpt-3