gjuggler
Joined 15 years ago
Joined 15 years ago
I think I recognize the strategy: most pharma environments connected to interesting data are tightly locked down, to the point where you can't just connect your Macbook to the source data.
Similarly, access to large genomic biobank datasets like UK Biobank or NIH's All of Us program is granted only through a Trusted Research Environment (TRE), a remote data analysis platform usually quite restricted on internet access, etc. You can't easily run desktop apps, but these environments do usually support running JupyterLab or VS Code, tunneling the user interface through to the end user. (Source: I previously ran the team that built the All of Us TRE.)
Claude Science looks a lot more like something one could imagine spinning up in one of those highly-constrained data environments (with the "server" running within the TRE and the UI proxied to the end user's browser) than the does-everything Claude mega-app. That will be critical for traction within pharma R&D environments.
I will say that for moderately-computational scientists, who are daily driving RStudio, JupyterLab, or maybe VS Code, Claude Science will be quite an unfamiliar shaped product. I'll be curious to see whether something like this gains adoption (1) in place of, (2) alongside, or (3) eventually wrapping around the more traditional data science workbench tools out there.
(We obviously live in a more nuanced world than most social media interactions might make you think :P)
> On a lighter note, decades ago, in middle school, we had an exercise to summarize a book we read.
My first experience with plagiarism was in first grade, when we were told to write a book report about a subject during our library time. I diligently took my book on the musk ox and copied three pages word-for-word into my notebook as my report. I can't remember when or how we learned this wasn't "right", but I still think back on that and laugh.
I started getting that "I'm reading another AI-written blog post" feeling around 1/3 of the way through, but I don't consider myself super calibrated on this.
Pangram seems pretty confident it's AI (https://www.pangram.com/history/e9f6eb77-86f9-46d0-a6c1-e57c...). But I know these tools aren't perfect. I'd love to hear from the author what their process was in writing this piece!
Related question (I'm trying to work this out for myself):
If you believe using AI to write an email or blog post for you isn't okay, but using AI to write code for you is... what's the difference?
Right now my instinct is something like:
- Code can be verifiably correct (especially w/ good tests) so it's less of a purely-creative act than writing.
- But always, always double-check the tests!
- I still wouldn't submit a PR where I can't vouch for every line of code.
- AI-written documentation and specs are mostly still bad and should be looked down upon. But mostly because the quality, at least today, is poor. (Lots of duplication, lack of a clear understanding of the reader's intent and needs, no thoughtful curation, etc.)
- Be psychologically ready to update these priors as models change.
I'd love to hear from anyone who's thought more about this.
Instead of forwarding model-generated links to https://www.google.com/url?q=[URL], which serves the purpose of malware check and user-facing warning about linking to an external site, Gemini forwards links to https://www.google.com/search?q=[URL], which does... a Google search for the URL, which isn't helpful at all.
Example: https://gemini.google.com/share/3c45f1acdc17
NotebookLM by comparison, does the right thing: https://notebooklm.google.com/notebook/7078d629-4b35-4894-bb...
It's kind of impressive how long this obviously-broken link experience has been sitting in the Gemini app used by millions.
So any data shared by this policy WILL absolutely be tied to the traditional methods of scientific credit, via the linked journal article. To me, requiring that reasonable data is published and archived alongside scientific literature doesn't seem absurd at all.
> Journals try to enforce data sharing because they want to maintain their position as arbiters of academic affairs.
Is there any evidence behind this claim? PLOS seems to behave in exactly the opposite way. It's true that SOME journals use extreme selectivity or control over copyright to maintain their position as arbiters of science. But PLOS, whose largest journal PLOS One is both open access and makes no judgment on the impact of the science it publishes, seems to be actively reducing the amount of control it exerts over academic activities.
My experience (in genomics / evolutionary genetics) was overwhelmingly positive, with huge amounts of collaboration all around. Scientists did try to publish at the highest-impact journal because they care about their career prospects, and there were often competitive labs racing to the finish line with a breakthrough publication.
But I never saw researchers withhold important details in order to accelerate their own publication or to hamper the efforts of others. By and large, I saw huge teams collaborating on large projects with a deep sense of purpose to move the field forward and improve understanding.
In fact research funding in genomics became so collaborative — with increasing portions of the research budget being put towards huge consortium-based projects — that smaller labs began criticizing funding agencies because their smaller projects weren't being funded. These projects are typically more competitive and higher risk, yielding potentially high-impact articles with a far smaller number of authors.
What's the answer in the end? As with many things, it's balance. Cooperation is great, but to a fault. Sometimes having small labs working in relative isolation, even competing against other small labs, can yield great innovation and progress. Other times you need a huge collaborative effort to do something big, expensive and important for the future of the field.
Let's say you release your project to GitHub & figshare and now have a DOI in hand. What are you supposed to do with it? Do you ask your users to cite this DOI if they use your software? If so, what text should accompany the citation? How do you track citations to your code? Will they show up in Google Scholar, Scopus, Web of Science?
And what if the journal one of your users is submitting to doesn't accept figshare / github citations? It's unfortunate but true that many publishers disallow citations to unpublished / non-academic works. This is why many scientific software projects have resorted to publishing papers on their software — it's a hack to make a software project fit into the traditional social system of scientific credit.
DOIs are a technical glue that binds together the thousands of academic publishing outlets, but they do not solve the scientific or cultural issue of what is the minimum viable citable scientific product, and how those citations are generated, propagated, or valued.
Securing a DOI only solves a small slice of the problem of scientific credit — a point most colorfully expressed by this blog post from CrossRef, the largest DOI registrar for academic work: http://crosstech.crossref.org/2013/09/dois-unambiguously-and...
But what surprised me most was that at the end of the essay, after having described his fear of facing such uncertainty in NIH funding, the author mentions that he left academia to co-found a startup making software for life scientists.
Wait a minute — don't small software startups have equally poor success rates? (e.g. http://www.quora.com/What-is-the-truth-behind-9-out-of-10-st...)
If uncertainty of success was his major concern, hasn't the author chosen a pretty poor next step in life?
On a related note, recently at http://paperpile.com/ we've been thinking about ways to help our users connect their reference manager with web-based writing environments like ShareLaTeX. We've had some ideas about how to do this simply and cleanly, but I suppose now we could show you guys (via pull request) rather than tell now. :-)