Yeah, that's a good point and I see where you're coming from. But generally when I hear the argument about consensus, it seems to end with nothing other than an implication of "and therefore they're obviously wrong, simply because of the current consensus".
But yeah, I see your point.