Comment

Comment by otabdeveloper4

Re point 1: no, "temperature" is not an inherent property of LLM's.

The big cloud providers use the "temperature" setting because having the assistant repeat to you the exact same output sequence exposes the man behind the curtain and breaks suspension of disbelief.

But if you run the LLM yourself and you want the best quality output, then turning off "temperature" entirely makes sense. That's what I do.

(The downside is that the LLM can then, rarely, get stuck in infinite loops. Again, this isn't a big deal unless you really want to persist with the delusion that an LLM is a human-like assistant.)

parent otabdeveloper4 11 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton