For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.
So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.
Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.
For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.
So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.
Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.