zaptrem

Probably would have just been Microsoft

Per hour per 8x node (so $1-1.25 per hour per gpu)

Per hour per 8x node (so $1-1.25 per hour per gpu)

Not the cheapest. Many providers have spot H100s for $8-12, and H200s for slightly less than the above prices

Can you explain how the above event doesn't count as evidence alignment is an actual risk?

Given these models could not have been trained in the first place if they had to license every line of random fan fiction on the internet, I think distillation also being fair game is a tradeoff everyone should be willing to take (unless they want to decelerate, but that's a different conversation).

This is my number one complaint about the M-series MBP line. Especially true of the cutout in the middle that has points so sharp they can cut you if you accidentally scrape it with your hand.

Why haven’t we seen any queues or the like over the past week then? If it’s truly a capacity limitation why not just boot subscription users to a lower priority queue or limit usage to outside peak hours?

Should we require the destruction of the brains of those that watch pirated movies?

Needs more WebGL spinning rubik's cube

Can you include GPT 5.5 non-pro (extra high thinking I guess) in your comparison? GPT Pro is the "I am willing to torch cash for a sooometimes slighty better result" option, not the one people are actually expected to use daily. That's probably part of the reason it's not in Codex

OOM on CUDA GPUs is relatively graceful (the process crashes). However, on macOS if torch MPS tries to allocate too much memory, the whole kernel will simply lock up and the only option is to reboot the computer. I have no idea why Apple doesn’t reserve memory for stuff like the OOM/kernel watchdog, but it seems they either don’t or there is a bug.

Love me some JSD. Here is a problem most people don't consider with generative modeling (e.g., AI text, image, music, video models): basically all standard pre-training algorithms for generative models (i.e., cross entropy, basically all diffusion/flow formulations) are closer to a Forward KL divergence. In other words, given limited capacity the model will try to stretch itself to cover every mode. This gives you a jack of all trades (lots of knowledge and diversity), but a master of none (you get blurry images and text filled with nonsense).

The real magic in generative modeling comes from the post training process that comes after, which usually (e.g., RLHF) approximates Reverse KL (given limited capacity, try to perfectly cover what you can, but it's fine to drop the rest entirely). This gives amazing results, but is also the cause of AI oddities like the "AI Image Pixar Look", many of the verbal tics of LLMs, and all AI music using the same small set of voices. Jensen-Shannon Divergence sits right in the middle of Forward and Reverse KL and is what many GANs are claimed to approximate. Ideally, it is a better trade-off between diversity and fidelity.

V4-Pro is about 2.4× total params and 1.3× active params of V3.2.

Seems pretty clear, Claude and Codex were getting a lot of free publicity by instructing their models to do the same and MS wanted similar results. However, a bug caused this to be applied to all commits instead of all Copilot-influenced commits.