Comment by raws

I feel like even a human would also fail if given all data results but those for x and then get tested on x just as the function results differ from previously observed behavior. Is it not more interesting to observe how the model(human or not) incorporates the new data to better match reality in the case of distribution shift or other irregular distributions and others