KaushikR2

See footnote 2:

> “Base models” are pretrained models that have not yet undergone reinforcement learning, SFT, or other post-training. They are highly sensitive to prompting, making sampling-based evals unreliable. Instead, we measured bits-per-byte loss on heldout data, which does not suffer from prompt sensitivity and smooths measurement of otherwise emergent abilities. As a side note, we were surprised that Nemotron 3 outperforms DeepSeek V4 Pro across the board but found this to be consistent across domains and inference engines. This might indicate that Nemotron’s weak performance on benchmarks after RL is due to weaker post-training, but the pretrain was ahead of Chinese open-weight competitors.

Perhaps parent was being sarcastic.

tptacek spotted

That's not always desirable though. I'd rather have control over that

And that's why it's important to vote! #BlueWhale2024

I wonder if it would be possible to setup a folder on a remote server, and use that as the directory for local files. Wonder if Spotify would play nice with that.

Optimization has taken a back-seat now, I'd say. We focus more on developer productivity, and that extra mile optimization isn't deemed necessary if that effort and time can be put toward another project, squeezing out more value from the developer.