WithinReason

Choosing the next token deterministically.

Apparently it's news to some people. It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.

What makes the process expensive? can you give a very rough breakdown?

The naming is elementary, my dear Watson.

too slow, already flagged

If you care about every last bit of performance, CPUs are difficult to program for as well, possibly more difficult than a GPU.

Generally GPUs punish you more by bad performance if you don't optimise your code.

Like nitrogen and oxygen gas that would take up the extra space inside?

Including training?

Interesting that they specifically nerf cyper-capabilities of open weight models:

"In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness."

The makers of Baldur's Gate, Star Wars: Knights of the Old Republic, Neverwinter Nights, Mass Effect, Dragon Age: Origins.

Sad, even though BioWare was dead already.

yes

To assert dominance