Comment by rico735

Why so sure they will get better and not just plateau? They seem to be doubling in cost each time with less notable gains, the economics seem increasingly shaky, and potential problems with model collapse and context window size has not been solved afaik.