Comment Comment by sillysaurusx Le sigh. Do I need to fix this sampler too? The default output should be excellent, not a drunk librarian.Rip open the code and make sure it has top_k 40, temperature 0.7, no top_p, and a repetition penalty of 1/0.85.Proof: https://twitter.com/theshawwn/status/1632569215348531201?s=6...Implementation: https://github.com/shawwn/llamaIt feels basically identical to davinci. I think no one realizes it yet because Facebook released a horrible sampler that everyone copied. parent sillysaurusx 3 years ago
Rip open the code and make sure it has top_k 40, temperature 0.7, no top_p, and a repetition penalty of 1/0.85.
Proof: https://twitter.com/theshawwn/status/1632569215348531201?s=6...
Implementation: https://github.com/shawwn/llama
It feels basically identical to davinci. I think no one realizes it yet because Facebook released a horrible sampler that everyone copied.