Comment by aprilthird2021

Isn't writing out the example in the beginning that Astra missed basically ensuring that every future model will get this task correct?

I guess my point is these benchmarks unless they are constantly refreshed with new scenarios and not ever having their outcomes written about publicly or available on a public server, will always be prone to being gamed