Comment

Comment by loveparade

It has always been like this in DL research. Also, providing justification doesn't necessarily help, because they are usually just guesses. The problem goes deeper than this. It's easy, and common, to make up justifications after the fact. You can try a bunch of random stuff, some thing randomly works, and then you make up an explanation for why it works that sounds plausible, but in reality is nothing more than a fictional narrative to convince reviewers (and possibly yourself). The underlying problem here are the incentives of the academic system. Reviewers reward good narratives and stories. Also, in DL especially, it's quite difficult to perform ablation studies or get statistical significance numbers due experiments taking a long time to run.

I hate to say it, because that's not how science should work, but with the flood of papers, looking at who wrote a paper is probably one of the more reliable indicators of quality. The reason is that people with a reputation are putting their reputation at stake when publishing something. Publishing BS would reflect badly on them, so they tend to go through more quality control. An unknown researcher at an unknown lab has nothing to lose by flooding arxiv with BS papers, which on the surface are indistinguishable from high quality work.

parent loveparade 2 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton