Yes, and none of the tutored students encounter the exact problems they’ll see on their own tests either.
In the language of ML, test prep for students is about sharing the inferred parameters that underly the way test questions are constructed, obviating the need for knowledge or understanding.
Doing well on tests, after this prep, doesn’t demonstrate what the tests purport to measure.
It’s a pretty ugly truth about standardized tests, honestly, and drives some of us to feel pretty uncomfortable with the work. But it’s directly applicable to how LLM’s engage with them as well.
In the language of ML, test prep for students is about sharing the inferred parameters that underly the way test questions are constructed, obviating the need for knowledge or understanding.
Doing well on tests, after this prep, doesn’t demonstrate what the tests purport to measure.
It’s a pretty ugly truth about standardized tests, honestly, and drives some of us to feel pretty uncomfortable with the work. But it’s directly applicable to how LLM’s engage with them as well.