Yes, we need to quantify tradeoffs between models mathematically, but that does not not require knowledge of the mathematics behind the models themselves. With cross validation, I can estimate the effectiveness of many black box models, without looking inside them. This step is called error estimation, and comes before model selection.
I can arrive at a pretty good model by a combination of correct methodology and brute force. It is this methodology that makes up much more of the overall picture. You could give me a black box, a rough range of parameters it takes, and I can tell you how likely it is to work well. This approach doesn't scale well to bigger problems, but I doubt tackling Big Data problems is the intention behind this course.
Yes, we need to quantify tradeoffs between models mathematically, but that does not not require knowledge of the mathematics behind the models themselves. With cross validation, I can estimate the effectiveness of many black box models, without looking inside them. This step is called error estimation, and comes before model selection.
I can arrive at a pretty good model by a combination of correct methodology and brute force. It is this methodology that makes up much more of the overall picture. You could give me a black box, a rough range of parameters it takes, and I can tell you how likely it is to work well. This approach doesn't scale well to bigger problems, but I doubt tackling Big Data problems is the intention behind this course.