rahulnair23

Of course, we don't wait to try it on real patients, so typically you'd partition the data you have already into (a) what you show to the machine learner (training data), and (b) what you hide from the learner (test data). The latter is only used to evaluate, i.e. you get the answer from the ML model and compare it to the real answer you have already. If information about the test data some how makes it to the training data, its referred to as a mining leak [1].

In this paper, they treat each month of a patient as an independent observation. However, GPS driver behaviour will be very similar from one month to the next for the same person. Genetic information is exactly the same. So for every month that the model is tested (test data), the learner has already seen very similar data in the training set - for some of the other months (for the same person) that happen to be in the training set. The split is typically done randomly. So it will do well.

The test results are therefore optimistic and do not support the conclusions.

> All models were trained on 70% of the data and tested on the remaining 30% of the data. Note that each month record for each participant was considered an independent data point.

I'm almost certain that this is a mining leak.

Data from one patient will end up both in the training and test set and result in fantastic accuracy. Of course it will be from different months. The correct way to do this is to cross-validate/split across the population. It seems unlikely, from this description, that the authors have done so.

Broad-based population screenings as the parent comment suggests, in my opinion, are.

I'm yet to see any clinically-valid distinguishing aspects that would suggest AI would add value to screening. Curious to hear evidence that drives your optimism of human+AI.

Just to state, the NELSON study [1] focuses on high-risk segments. Their paper also recommends a "personalized risk-based approach" to screening. This seems reasonable.