This is almost always a very bad idea because the if/then condition you describe is
business logic - as in, how do you recognize the
business situation when you want to decline a prediction, and how does that change?
It is very complex because most of the time there is no simple rule such as a threshold on the confidence score of the prediction. In practice it might be more like, “if the user has more than 7 items in their cart and if the user is not a returning customer that filled out personal data and the value of their cart is greater than $100 and they have not put a new item in the cart for 2 minutes, and the confidence score of the predictor is less than 0.4, THEN don’t show the next recommended item, just display a checkout link.”
And the number of items, the cart value limit, the time since last item-add, etc., will all be hotly debated by product management and changed 5 times every quarter.
In grad school there was a professor who said of machine learning that “parameters are the death of an algorithm” - so you want to avoid coupling extra business logic parameters tightly with the use of machine learning models.
It is very complex because most of the time there is no simple rule such as a threshold on the confidence score of the prediction. In practice it might be more like, “if the user has more than 7 items in their cart and if the user is not a returning customer that filled out personal data and the value of their cart is greater than $100 and they have not put a new item in the cart for 2 minutes, and the confidence score of the predictor is less than 0.4, THEN don’t show the next recommended item, just display a checkout link.”
And the number of items, the cart value limit, the time since last item-add, etc., will all be hotly debated by product management and changed 5 times every quarter.
In grad school there was a professor who said of machine learning that “parameters are the death of an algorithm” - so you want to avoid coupling extra business logic parameters tightly with the use of machine learning models.