anishrverma
Joined 4 years ago
Karma
9
Submissions
15
Member since
Jun 4
Recent submissions
Comment · 6 months ago
Comment · 6 months ago
Agreed. Accountability matters, but changing the game usually scales better than hoping for better individual behaviour under the same pressures. Academia needs systems that reward transparency, verification, and contribution more directly. That is part of what we are building with Liberata if of interest: https://liberata.info/beta-signup
Comment · 6 months ago
This is what makes the problem feel so systemic, in that weak consequences after the fact, and weak incentives for transparency before the fact. If the system mostly rewards output and prestige, then misconduct can remain a high-upside bet. We should be building research infrastructure that makes review trails, contribution, and verification more visible much earlier. That is part of what Liberata is aiming at, if of interest: https://liberata.info/beta-signup
Comment · 6 months ago
I think this is the right tension, in that bad incentives matter, but that does not remove personal responsibility. We probably need both stronger accountability for clear misconduct and better systems that make rigor, transparency, and verification easier to pursue in the first place. The second piece gets much less attention than it should. That is a big part of what we’re trying to tackle at Liberata: https://liberata.info/beta-signup
Comment · 6 months ago
This (also) feels like a core failure mode, in that papers are optimized for skim-level persuasion because the system is too overloaded for deep evaluation at scale. Then a lot of the actual scrutiny gets pushed onto under-credited sub-review labour. Peer review is too important to stay this invisible and under-incentivized. Liberata is exploring exactly that problem, and our beta waitlist is open if you want to follow along: https://liberata.info/beta-signup
Comment · 6 months ago
This feels like a core failure mode: papers are optimized for skim-level persuasion because the system is too overloaded for deep evaluation at scale. Then a lot of the actual scrutiny gets pushed onto under-credited sub-review labour. Peer review is too important to stay this invisible and under-incentivized. Liberata is exploring exactly that problem, and our beta waitlist is open if you want to follow along: https://liberata.info/beta-signup
Comment · 6 months ago
Strongly agree. When code, data, and workflow details stay hidden, the system rewards claims more than verification. That is where shortcuts, irreproducibility, and worse can thrive. We need infrastructure that gives more credit to transparency and reusability, not less. That is part of what we’re building at Liberata if you’re curious: https://liberata.info/beta-signup
Comment · 6 months ago
I think this is exactly the hard part: individual virtue alone does not solve a system where supervisors, trainees, and funders are all pulled by the same incentives. "Do slower, better science" is not actionable unless the surrounding infrastructure and rewards change too. That is a big part of what we're thinking about with Liberata, especially around peer review and attribution. If relevant, our beta waitlist is open: https://liberata.info/beta-signup
Comment · 9 months ago
I don’t read the NeurIPS statement as malicious per se, but I do think it’s incomplete
They’re right that a citation error doesn’t automatically invalidate the technical content of a paper, and that there are relatively benign ways these mistakes get introduced. But focusing on intent or severity sidesteps the fact that citations, claims, and provenance are still treated as narrative artifacts rather than things we systematically verify
Once that’s the case, the question isn’t whether any single paper is “invalid” but whether the workflow itself is robust under current incentives and tooling.
A student group at Duke has been trying to think about with Liberata, i.e. what publishing looks like if verification, attribution, and reproducibility are first class rather than best effort
They have a short explainer here that lays out the idea if useful context helps: https://liberata.info/
Comment · 9 months ago
Yeah, spot on. If all we do is add more plausible sounding text on top of already fragile review and incentive structures, that really could make things worse rather than better
Your second point is the important one. AI may be the thing that finally forces the community to take reproducibility, attribution, and verification seriously. That’s very much the motivation behind projects like Liberata, which try to shift publishing away from novelty first narratives and toward explicit credit for replication, verification, and followthrough. If that cultural shift happens, this moment might end up being a painful but necessary correction.
Comment · 9 months ago
Agreed.
What I find more interesting is how easy these errors are to introduce and how unlikely they are to be caught. As you point out, a DOI checker would immediately flag this. But citation verification isn’t a first-class part of the submission or review workflow today.
We’re still treating citations as narrative text rather than verifiable objects. That implicit trust model worked when volumes were lower, but it doesn’t seem to scale anymore
There’s a project I’m working on at Duke University, where we are building a system that tries to address exactly this gap by making references and review labor explicit and machine verifiable at the infrastructure level. There’s a short explainer here that lays out what we mean, if useful context helps: https://liberata.info/
Comment · 9 months ago
The prevalence of hallucinations in the system is another signs for change in the system. The citations should be treated less like narrative context and more like verifiable objects
Better detectors, like the article implies, won’t solve the problem, since AI will likely keep improving
It’s about the fact that our publishing workflows implicitly assume good faith manual verification, even as submission volume and AI assisted writing explode. That assumption just doesn’t hold anymore
A student initiative at Duke University has been working on what it might look like to address this at the publishing layer itself, by making references, review labor, and accountability explicit rather than implicit
There’s a short explainer video for their system: https://liberata.info/
It’s hard to argue that the current status quo will scale, so we need novel solutions like this.
The broader goal is to rethink academic publishing around better incentives: more granular contribution attribution, rewarded peer review and replication, and graph-based metrics for impact, risk, concentration, and possible collusion.
We would love feedback from people interested in scientometrics, citation networks, peer review incentives, mechanism design, or open science.
Beta signup: https://liberata.info/beta-signup