0d9eooo

But it also fuzzes value. So what you end up with rather than "value" is network effects, people who are good at advertising, and random chaotic trends that blow like the wind. That is, it's not just enough to be the best, it's required to create the appearance of the best, or to capture whatever the whims of the network are.

People are also not always very good judges of long-term value. For the longest time the response I'd get whenever I expressed some sentiment of supporting small business, all things being equal, was something along the lines of "well you should support whoever can deliver the good at the lowest price." Although that's true to some extent, in the long term it leads to personal costs as local services are gone, and there's less and less competition. Short-term benefits, long term costs and so forth.

In general, I think there's far too little focus on maintaining healthy competition. The pendulum in public discussions often swings from public to private services and employment, which is a good conversation to have, but there's almost never a serious, critical discussion about increasing competition and what those barriers are. US society is full of things that stifle competition and they don't get 1/10 the attention they deserve. It seems as if when it's brought up, you're forced into these ultra-libertarian or anti-capitalist stereotypes.

The other day I was looking for a cycle restoration shop. There's a great one and they should be thriving right now. But their business was destroyed in riots. Not the business owner's fault. You could make an argument that it's just one more downstream effect of a racist cop killing someone over a counterfeit bill, a very significant one. This effect has played out many many times across the country now I'm sure. But who will foot the bill? The Mpls police dept?

It seems we used to have this intuitive understanding of "bad things happen to good people" and "the community should pitch in collectively to support these things" but it's fading away.

The darwinian model is flawed, unless you're focuses solely on the net benefits to those who benefit, which is circular in its reasoning.

I sincerely appreciate this blog post and discussion, as it raises a lot of important and compelling points, but I wish it had had some discussion of alternatives and why.

To me, messaging is a mess at the moment, somewhat like IoT because of lack of solid widely adopted standards (either de facto or de jure).

It's extremely difficult to get friends and family to use something. Most decisions are driven by secondary considerations, like it comes with an OS, or as part of an email or office system, or a gaming system. In some cases it's because "it's what everyone is using".

This shifts the threshold a bit in terms of concerns. What I mean by that is given the inertia involved in moving people to use a messaging system, the bar gets raised in terms of moving people off because of network effects. It's hard enough to get any friends or family to use Signal as an alternative to other things; convincing them to switch again introduces other problems.

I'd prefer something that can be used in more decentralized way, but that has its own issues in terms of syncing and always-on problems. And as security increases, more and more inconveniences are introduced -- it might be worth it, but the case still has to be made implicitly or explicitly to friends and family.

Again, not saying these kinds of discussions shouldn't happen, but they often seem kind of theoretical to me or like they're missing the point because of bigger issues with the messaging ecosystem in general. If you're not going to be able to use Signal anyway because everyone you know is using Whatsapp or iMessaging, or feel like messenger use is driven by "whatever is most popular" it feels like it's difficult to weigh things like "won't put on fdroid". I'd love to see it on fdroid but where does that rank?

It depends on what you mean by "science."

Science has become so political and subject to hype and fads, at least in biomedicine, that everyone should be deeply skeptical of every claim that is made. That doesn't mean that you should distrust everything completely, just that critical skepticism should be the norm.

If by "science" you mean that "rigorous, logical empirical study" should be part of the decision-making, yes, I agree.

If by "science" you mean that the statements of scientists with prestigious academic appointments should be given priority over other considerations, I say no. Outsider criticism should be welcome, and I think nonscientific considerations are sometimes required.

One of the reasons society is in the mess it is in (at least in the US) is because of a failure to realize that certain segments, like the biomedical-scientific community, are just as driven by human failures as any other.

It's already too political. I agree with you; just saying that everything bad in that regard will get even worse.

I feel odd being so gushing about Best Buy, because for the longest time I really didn't have the most positive impression of them at all, but I'm not exaggerating when I say that over the last few years, I've had the best customer service experience from them that I've had anywhere in any store of any sort. It's really phenomenal, like enough to make me make a point of purchasing things there when I have the option. I don't even know where to begin, but I feel like they've been super attentive and genuinely helpful without dumbing things down.

Whether that's worth it or not to any given person is a personal thing but I was really surprised by it.

10 million confounds is a lot to eliminate. Randomization only works because it decreases the probability of a spurious association existing. At some point the number of confounds eliminated would probably be more worthwhile. I'd definitely pay more attention to a study that eliminated 10 million confounds than most randomized controlled trials.

I agree a randomized controlled trial would be nice now. But even that is potentially fraught. For example, someone else mentioned the possibility that glucosamine reduces joint pain, which increases mobility, which increases longevity. Randomizing wouldn't really control for that sort of scenario. And that's not even getting into preregistration, meta-analysis etc.

I think the horse has left he barn in terms of the suggestions in the comic. I love the comic and the attention it's getting but the issues discussed are such a small part of the problem. Many of the comments here are on point.

At this point in time, many of the people in power (full professors, administration, etc) see papers as essentially useless, in part because of all the problems being discussed here. p-hacking, guest authorship, etc and so forth and so on. They're seen as a dime a dozen and somewhat ignored.

This might sound good until you understand that this means that they're dismissing the entirety of the scientific dialogue essentially, and that their alternative is grant money. My institution had training workshops for grad students where faculty would tell students that research is essentially worthless unless grant money is attached to it. The idea is that papers are a dime a dozen, and that if something is worthwhile, the feds will put their money where their mouth is so to speak.

The problem with this, of course, is that the grant system is horribly nepotistic and distorted. The biggest predictor of successful grant application last time I looked (based on empirical research in peer-reviewed journals) is co-authoring papers with someone on the review panel. Grant receipt is only weakly related to citation metrics, and from personal experience I can say that institutions are constantly encouraging researchers to inflate grant costs to bring in more indirect costs.

You could fix everything about journals and the publication process and it would do nothing about the shadow scientific world that exists in parallel that drives all the rest of the problems.

Eliminate indirect funds, require tenure of all researchers, set aside funding mechanisms for researchers that aren't tied to specific grant applications, randomize grant rewards, depriortize journals, ... there's a lot of things that need to happen.

IIRC, Hungary has implemented something like this and it's been floated as an idea, to distribute grant dollars as a fixed amount to everyone who has an H index or citation count above a certain amount.

When I've talked about this with colleagues, they've argued this would make problems worse, and it might, but I think the idea was more that once you reach a certain threshold you should be funded some amount.

Of course, I'd argue that this is basically the idea behind tenure at an R1 institution, that once you reach a certain level the state or private board of trustees is paying your salary to do research. But nowadays tenure at an R1 institution is proxy for external grant dollars, which defeats the purpose and is redundant.

The real problem is indirect funds, which create profit for institutions. The federal government needs to eliminate indirect funds, so that grant dollars aren't perversely incentivised and tenure is based on research quality rather than profit.

Maybe I read the original research article incorrectly, or maybe this BBC piece is discussing something additional, but my impression after reading about it more closely was that this test is actually discriminating Alzheimer's from other forms of dementia, not Alzheimer's vs no dementia.

I haven't thought this through yet but it seems interesting and important but also not quite how it's being portrayed in the media, and not quite as significant in some ways.

I agree that business isn't really the right model for academics but the problems imho is that what an "outstanding academic track record is" has become so ambiguous that it's practically meaningless -- and I say this as a former tenured prof at an R1 institution.

I could write a book about this stuff. The stories I could tell about what's behind those "outstanding academic track records"...

The problem, if anything, is trying to apply a business model to academics, equating research quality with federal grant dollars, taking away real intellectual freedom protections, and then ignoring all the ponzi scheming and exploitation that occurs. Everyone has their heads in the sand, knows academics (at least biomedical research) is full of BS, and just goes on pretending like it's not because no one knows of a good alternative, or doesn't have the courage or power to change things.

What's funny [sad?] to me is that your description of PMs sounds exactly like the most credentialed, accomplished researchers I know on paper.

It's interesting to me regarding some of the examples in the linked piece. Ghostwriting reviews, for example, is actually seen as a good practice in a lot of circles because it provides experience to grad students with the review process. Those guest authorships? Very grey area between that and collaborative authorships. It's not the grunt work, it's the idea, right? Or is it that ideas are a dime a dozen, and actually doing the work is important? I can't tell which it is anymore -- it seems to depend on what benefits those in power.

Someone else posted something about how 1% of research is fraud, and 80% is bad. I think the percent of fraud is probably higher, the percent bad research is lower, and the difference is much more fuzzy than you'd think initially. The really difficult thing is that tiny incremental contributions is how things actually work. No one wants to admit this though. Bad research is actively incentivized, and there's credit bubbles everywhere.

The worst problem is that this credentialing bubble is everywhere with everything, as another posted noted. The problem isn't the credentialing per se, it's how it's detached from reality, the real demands of the tasks. Having a credential doesn't mean that the person is competent for all the tasks it nominally encompasses; conversely, those tasks don't necessarily require the credential that's often demanded.

The harder story to tell that this piece is avoiding because of the difficulty is the institutional dynamics. It's hinted at in the article more or less directly, but the entire mindset at many places is so far from what's conveyed in this New Yorker piece that it sort of misses the ball completely.

At the time the story in this article begins, administrators at many places were completely unfamiliar with the general idea behind open content distribution. In such places, online systems for course content were seen as a way of locking down the intellectual property that courses represent, not to distribute it. Where I was at, faculty were explicitly admonished not to distribute any online course material because it was seen as leaking university IP. Course management systems were seen as a way to deliver materials online securely so as to ensure that they weren't widely distributed. Even if someone wanted to put the effort into something that might now be called a MOOC, it was doubly frowned upon because you were putting effort into teaching rather than research (especially if you were untenured), and triply so because you were putting effort into distributing courses online, which was a fringe activity.

By the time MOOCs came about and got a lot of press, and many of us were sort of shaking our heads at what seemed inevitable, it was a little too little too late. In online space, where social factors like reputation become amplified 100-fold, it's difficult to compete or add to the reputational weight of places like MIT or Harvard. Adding to this was the constant monetizing mindset, of trying to lock down materials, not recognizing the advertising/popularizing role that open content distribution enables. Then too was a sort of "so what" feeling, because in the absence of something like a pandemic, who really cares, from the perspective of the administration -- students come to campus and that's where the dollars come from; MOOCs and online course distribution of whatever form were seen as experimental, the realm of the extension service which is valued but only to a point.

There's lots of issues the piece doesn't get into, or only gets into in a half-hearted way, which is how these types of online teaching dynamics really mess with the traditional structures. So instead of having an interesting conversation about how unfamiliar it is for universities to be dealing with Course A at Institution 1 competing with Course B at Institution 2, it recapitulates the discussion in terms of Malan, which misses the bigger picture entirely. These issues are not about one instructor of one course at one university.

Maybe I'm just getting jaded, but I feel like this piece does a disservice to the behemoth of problems at universities represented by the online instructional debacle we've witnessed. Like lots of things, the pandemic is exposing widespread systemic problems, not creating them.

Peer review is a cornerstone of academics, and there continues to be a prestige associated with it as well as with certain journals. This is especially true in certain circles.

As far as I can tell though, functionally this is breaking down. People can find preprints and archived papers, and do, if they're searching by topic.

So journals at this point are providing a peer review portal, and formatting. I happen to think the formatting does provide value. My sense is that at a good journal, there's a kind of stochastic improvement in errors and formatting, so that the numbers of errors go down on average, and the formatting improved, on average, over iterations back and forth with the copyeditor.

As for peer review, I'm not so sure anymore. My sense is that it does provide some kind of stamp of approval from experts in the area, so if you don't know much about an area, it provides some sense that at least some small group of people in the area believe it meets some kind of basic standards. But that says very little, and the amount of noise in the review process is large.

I think the core of the academic communication system is slowly being hollowed out, and being replaced by blogs, things like twitter and mastodon, and archives. At this point the peer review journal process provides some value, but it's being propped up by tradition. Already, with COVID, we're increasingly seeing the focus on preprints. Journalists and others are careful to note something hasn't been peer-reviewed yet, but everyone knows it matters little because they can turn to experts to find out what they think of it.

If there aren't formal attempts to create an alternative, I think we'll just be left with people posting and passing around preprints and discussing them on twitter, mastodon, blogs, and message groups. If people want the nice formatting, and some stamp of approval, I think something else will have to be worked out. But the journals are starting to feel like they're getting in the way, in general, and represents some kind of power or status structure more than quality control system.

Paying reviewers I think creates bad incentives as the author of the post points out. So do author-pays systems. What is maybe missing from the piece is some recognition that in the past, reviewers reviewed and editors edited in part as part of their job. That is, you were paid as a faculty member at a university, and that was what people understood you did. Pre-internet, this was all valuable service. Now that universities and others are more focused on faculty bringing in profits rather than paying for their services -- and questions are being raised about the value of journals in general -- we are seeing these questions about what reviewers get paid.

I think in the future there will be value in article hosting and searching, and providing website frameworks for discussion and peer review, but I'm not sure they will look like journals per se. You'll see things like arxiv.org, but with commentary, rating, discussion, and approval infrastructure over them. That's what large libraries and research centers will be donating money to or paying for. I think journals per se will eventually start to seem kind of stodgy and old fashioned.

It's not just here. It seems like a common sentiment in a lot of places. It's confusing to me as well. There's a direct line in my mind from calls for social media to be self-regulate or be regulated (interestingly in different ways from conservatives and liberals), to these kinds of comments from Erdogan. It might be causal, it might just create a cultural context, but it's hard not to conclude that echoing calls for regulation of social media in the US aren't being coopted or leveraged in this case in a way that's very predictable.