My experience like this was for a data scientist position at Facebook in 2022.
So the question was something like, here is a table of predicted probabilities for credit card transactions (hypothetical table, I do not remember the exact quantities), but it looked something like this:
Amount, Probability, ActualFraud
$5 , 0.2 , 0
$100 , 0.5 , 0
$3 , 0.7 , 1
$1 , 0.9 , 1
And then the question was "what was the threshold used to maximize (precision|recall)?" (I cannot remember which metric they were asking for).
My response was that a proper decision model should take the dollar amount of the transaction into consideration, so the decision should more like be `$*p > threshold`, where this threshold meets your precision and recall for preventing dollars lost instead of the binary yes/no.
The interviewers English was not great, so after all that I was just like "well to answer your exact question, it could be anywhere between `0.5 < p <= 0.7`"
Did not make the second round!
So the question was something like, here is a table of predicted probabilities for credit card transactions (hypothetical table, I do not remember the exact quantities), but it looked something like this:
And then the question was "what was the threshold used to maximize (precision|recall)?" (I cannot remember which metric they were asking for).My response was that a proper decision model should take the dollar amount of the transaction into consideration, so the decision should more like be `$*p > threshold`, where this threshold meets your precision and recall for preventing dollars lost instead of the binary yes/no.
The interviewers English was not great, so after all that I was just like "well to answer your exact question, it could be anywhere between `0.5 < p <= 0.7`"
Did not make the second round!