I think one of the big things that you can do is split up the data gathering and data analysis.
Kind of like we don’t trust the companies to audit themselves, instead we have an outside firm.
In this model, a researcher would create a hypothesis and collect the data. Their team would write the background and
methods sections of the paper.
Then the entirety of raw data would be sent off to a third party for data analysis and they would write the results part of the paper.
The original team would then write the discussion part discussing the implications of the study.
All papers would be required to be made public.
The idea is that there would be specialized firms that do the analysis on the raw data for everyone. These would be carefully audited and certified by the government. They have no incentive to play statistical games. If they get caught cheating, then they have to pay for all the analysis on all the papers to be done again by a competitor and if any errors are found in a paper analysis it is automatically retracted. While the this re-analysis is going on, all papers would be quarantined with a note stating the paper is having its analysis redone. This incentivizes the analysis firm to be ethical, as well as incentivizes researchers to pick ethical analysis firms.
Separating data collection from data analysis would help align incentives better.
Kind of like we don’t trust the companies to audit themselves, instead we have an outside firm.
In this model, a researcher would create a hypothesis and collect the data. Their team would write the background and methods sections of the paper.
Then the entirety of raw data would be sent off to a third party for data analysis and they would write the results part of the paper.
The original team would then write the discussion part discussing the implications of the study.
All papers would be required to be made public.
The idea is that there would be specialized firms that do the analysis on the raw data for everyone. These would be carefully audited and certified by the government. They have no incentive to play statistical games. If they get caught cheating, then they have to pay for all the analysis on all the papers to be done again by a competitor and if any errors are found in a paper analysis it is automatically retracted. While the this re-analysis is going on, all papers would be quarantined with a note stating the paper is having its analysis redone. This incentivizes the analysis firm to be ethical, as well as incentivizes researchers to pick ethical analysis firms.
Separating data collection from data analysis would help align incentives better.