whodatbo1

Sooo, this implies such deep profiling hasn't been in use for a decade for target advertisements. 500M is not that large a number with the amount of traces we leave behind online :)

A better metric is price per byte. Most thinking traces, prompts, skills are in plain English, which is roughly 1 byte per character, assuming UTF-8 encoding (even code should not be much more either). As an aside, it is common to use bits-per-byte as a loss metric instead of the per token calculation, precisely because of the effect of different tokenizers.

The issue here is that you can never be sure how the model will react based on an input that is seemingly ordinary. What if the most likely outcome is to exhibit malevolent intent or to construct a malicious plan just because it invokes some combination of obscure training data. This just shows that models indeed have the ability to act out, not under which conditions they reach such a state.

benchmaxing is the expectation ;)

Absolutely agree. Unironically through the first few paragraphs I though this was some very cleverly put satirical take on modern society. There is a great book about the effects of modern food, primarily on dental health, written back in the 30s - Nutrition and Physical Degeneration by Weston Price. Even though it's almost a century old, the findings are quite fascinating and can be extrapolated to present day. It's quite undeniable that modern life has drawbacks and obesity is just a symptom.