Comment by apwheele

While this is true, it is still possible to take the probabilities and make determinations of recall and false positives using conformal sets, https://crimede-coder.com/blogposts/2026/ConfClassification

(Or just use a model to re-calibrate the probabilities, I like the conformal approach though as those rates are what I often care about.)

You just need some labelled data to generate the "corrected" probabilities (or thresholds to meet the specified error rates).