Comment by atleastoptimal

GPT-4 is 1.8T parameters. Until a model with at least 2-4x the number of params with no discernable improvement is released can we say that improvement has stalled. The biggest I imagine are Opus and Gemini Ultra with around 2T, far less that the geometric increase to demonstrate new emergent capabilities. Either way evidence has not been put forward yet to disprove scaling laws so until then, the apparent stalling is simply that a next-gen OOM model has yet to be trained and deployed.