I would have liked to have seen the underlying token count, and the full responses as an optional filter. My understanding is that under the hood the GPT-5 response (as well as omitted O model responses) would end with the presented output, but would have had many paragraphs of “The user wants X, so I should try to Y. Okay this z needs to be considered” etc.
It doesn’t detract from the progress, but I think it would change how you interpret it. In some ways 4 / 4o were more impressive because they were going straight to output with a lower number of tokens produced to get a good response.
It doesn’t detract from the progress, but I think it would change how you interpret it. In some ways 4 / 4o were more impressive because they were going straight to output with a lower number of tokens produced to get a good response.