Please, no. While he provides some food for thought, asks a few important questions (and provokes even more), this text is full of pseudointellectualism. As in: fancy words but utter lack of understanding of the core subjects one writes about. Most importantly: what is science (no, it is not a trivial question; I recommend going Ludwik Fleck's "Genesis and development of a scientific fact" route,
http://www.evolocus.com/Textbooks/Fleck1979.pdf) and what is AI (it is a vast field; some of it IS math/CS (as in: proving things), but it is a small part).
For example, many of things he says would fit other practical disciplines, e.g. medicine. Yes, experiments are on a group of people. Yes, criterion that "drug X works better than drug Y, but we don't know why" is sufficient.
> I don’t know data science folks well, but my impression is that they find the inexplicability and unreliability of AI methods frustrating.
Well, it is not the main issue (speaking as a data scientist working with AI). At least he acknowledges his lack of expertise.
> These failures of scientific practice seem as common in AI research now as they were in social psychology a decade ago. From psychology’s experience, we should expect that many supposed AI results are scientifically false.
Also - no, it is not at level of psychology when it comes to the replication crisis. A lot of code (though, unfortunately, not all) is shared online, by the authors or other contributors, and people do replicate it (or there is an absence of replication, which also conveys a message).
For example, many of things he says would fit other practical disciplines, e.g. medicine. Yes, experiments are on a group of people. Yes, criterion that "drug X works better than drug Y, but we don't know why" is sufficient.
> I don’t know data science folks well, but my impression is that they find the inexplicability and unreliability of AI methods frustrating.
Well, it is not the main issue (speaking as a data scientist working with AI). At least he acknowledges his lack of expertise.
> These failures of scientific practice seem as common in AI research now as they were in social psychology a decade ago. From psychology’s experience, we should expect that many supposed AI results are scientifically false.
Also - no, it is not at level of psychology when it comes to the replication crisis. A lot of code (though, unfortunately, not all) is shared online, by the authors or other contributors, and people do replicate it (or there is an absence of replication, which also conveys a message).