I have a favourite test for LLMs that is also still surprisingly not passed by many:
You walk up to a glass door. It has 'push' written on it in mirror writing. What should you do and why.
Very few can get it right, even fewer can get it right and explain the right reason. They’ll start going on about how mirror writing is secret writing and push written backwards is code for pull, rather than just that it’s a message for the person on the other side.
You walk up to a glass door. It has 'push' written on it in mirror writing. What should you do and why.
Very few can get it right, even fewer can get it right and explain the right reason. They’ll start going on about how mirror writing is secret writing and push written backwards is code for pull, rather than just that it’s a message for the person on the other side.
No version of Gemini has ever passed.