I manage an applied research group in the machine learning space, particularly AutoML and synthetic data generation. That means is we read the latest literature and create applications. Sometimes we build new research, most of the time we patent something new. But the goal is to get research to production as fast and as impactful as possible.
The first thing I do when sorting research (papers, blog posts, presentations, etc) is sort the research by “B.S.”, “maybe B.S.”, “probably real”.
If you look at research papers in computer science, half are what have the following problem: “running a simulation over and over until the authors get the results They like”.
I can’t tell you how many papers with thousands of citations are just blatantly wrong. They don’t run corrections based on the number of simulations they run, they don’t take into account other variables, etc
This isn’t limited to CS either. Biology, economics, environmental sciences, etc all suffer the same fate.
The worst part about it, is it’s PEER REVIEWED. Meaning, others agree this is the way to do things, which is why I don’t trust academia almost at all.
The first thing I do when sorting research (papers, blog posts, presentations, etc) is sort the research by “B.S.”, “maybe B.S.”, “probably real”.
If you look at research papers in computer science, half are what have the following problem: “running a simulation over and over until the authors get the results They like”.
I can’t tell you how many papers with thousands of citations are just blatantly wrong. They don’t run corrections based on the number of simulations they run, they don’t take into account other variables, etc
This isn’t limited to CS either. Biology, economics, environmental sciences, etc all suffer the same fate.
The worst part about it, is it’s PEER REVIEWED. Meaning, others agree this is the way to do things, which is why I don’t trust academia almost at all.