radioactivist

I am somewhat skeptical of this.

First, the headline result of 0.7*sigma improvement is the output of a statistical based on lessons/reviews they engaged with and their mid-term score, with that shift being for "full engagement". Based on their tables something like ~16 students (11% of the group) actually reached that level of engagement

Second, trying to incorporate past grades into their modelling is not a substitute for a randomized trial.

Third, the headline engagement number of 90% is for "engaging with the platform, via Module Review or Lesson Quizzes, at least once". I don't know why much of that couldn't just be attributed to novelty. Or even partly a professor with all sorts of enthusiasm for the platform.

Fourth, the "full dosage" effectiveness is measured based the final exam scores. Were these exam questions produced independently from the "Phosphor" materials? (e.g. by blinding?) Were they checked for direct overlap with those materials? The 0.7 sigma shift is 3 points on a 24 point exam; if even a few of the questions on that exam were very similar to those materials it could account for almost all of it. This is not clear to me from the manuscript.

If this was the case, then it's a question less of "is AI effective" vs. "did the students look at the materials". You could still argue that the AI platform got them to read, but that is a somewhat different statement than the AI helped them learn.

While I echo some of your points, [1] is bad example (as a Canadian).

Research money in Canada is harder to come by; a basic research grant is roughly ~5x-10x lower than a comparable American grant (students are cheaper here, so its not completely proportional, but equipment, travel, etc doesn't scale).

The example for money for poaching international researchers also comes with the asterisk that while they found ~$2B for this, they also are cutting the base funding of the federal granting agencies by a few percent at the same time, atop of that funding being anemic for decades at this point. A big "fuck you" to the Canadian research community in my opinion.

Also a physicist here -- I had the same reaction. Going from (35-38) to (39) doesn't look like much of a leap for a human. They say (35-38) was obtained from the full result by the LLM, but if the authors derived the full expression in (29-32) themselves presumably they could do the special case too? (given it's much simpler). The more I read the post and preprint the less clear it is which parts the LLM did.

Thanks! (very quickly too)

Is anyone else having trouble using even some of the basic features? For example, I can open a comment, but it doesn't seem like there is any way to close them (I try clicking the checkmark and nothing happens). You also can't seem to edit the comments once typed.

In my circles the killer features of Overleaf are the collaborative ones (easy sharing, multi-user editing with track changes/comments). Academic writing in my community basically went from emailed draft-new-FINAL-v4.tex files (or a shared folder full of those files) to basically people just dumping things on Overleaf fairly quickly.

Seems like the someone dug something up from the literature on this problem (see top comment on the erdosproblems.com thread)

"On following the references, it seems that the result in fact follows (after applying Rogers' theorem) from a 1936 paper of Davenport and Erdos (!), which proves the second result you mention. ... In the meantime, I am moving this problem to Section 2 on the wiki (though the new proof is still rather different from the literature proof)."

This is a comparison between a new and interactive medium (+ slides, mind-maps, etc) and a static PDF book as a control. How do we know that a non-AI based interactive book wouldn't give similar (modest) increases in performance without any of the personalization AI enables?

Thank you for this comment, it is exactly my impression of all of this as well.

It happens again in the next video. It says:

> The team came up with a use case the teaching team hadn’t thought of – using AI to critique the team’s own hypotheses. The AI not only gave them criticism but supported it with links from published scholars. See the demo here:

But the video just shows Claude giving some criticism but then just says go look at some journals and talk to experts (doesn't give any references or specifics).

At one point this states:

> Claude was also able to create a list of leaders with the Department of Energy Title17 credit programs, Exim DFC, and other federal credit programs that the team should interview. In addition, it created a list of leaders within Congressional Budget Office and the Office of Management and Budget that would be able to provide insights. See the demo here:

and then there is a video of them "doing" this. But the video basically has Claude just responding saying "I'm sorry I can't do that, please look at their website/etc".

Am I missing something here?

I'm not the person you're replying to, but in my subfield (scientist is such a broad term) I would say in my opinion at least half of those key problems that are listed in the article are basically non issues. Things really are quite different field to field.

And in many subfields there is a preprint freely available on the arxiv during those three months.

OpenAI o4-mini-high

   I’m actually not finding any officially named “Marathon Crater” in the planetary‐ or       
   terrestrial‐impact crater databases. Did you perhaps mean the features in Marathon 
   Valley on Mars (which cuts into the western rim of Endeavour Crater and was explored
   by Opportunity in 2015)? Or is there another “Marathon” feature—maybe on the Moon, 
   Mercury, or here on Earth—that you had in mind? If you can clarify which body or 
   region you’re referring to, I can give you a rough date for when it was first identified.

Small correction: The meow meow beans episode of Community aired in 2014 and the Nosedive episode of Black Mirror aired 2016. So the Community episode came first.