kenferry

This paper doesn't say ANYTHING about _capacity_ for critical thinking, and by using the words "critical thinking" yes it makes it sound like "more is better", but look at what they're actually doing. What they're measuring I would much more call "oversight".

If I want to know when Obama was born, and check wikipedia and believe what it says, by their measure I am practicing less critical thinking than if I found independent verification. Independent verification is the main way they measured critical thinking. They are NOT measuring that I thought about how much I could trust the result, and decided that yes I could.

And, well, yes, they're right. But for most purposes, wikipedia is pretty credible for something like that, and it's just a waste of time to follow up on it. What's important is that my calibration is correct.

I'm not even saying I disagree with the idea that there is likely a problem here, with people offloading thinking. But I'm thinking critically about this particular study, which seems totally useless.

Hoo, not been my experience.

AI has an obvious issue with incorporating how happy users are in its responses. It LOVES to "take your argument seriously", push back lightly, and then tell you you are really smart and make amazing points. If you start with an argument out of left field, you will shift the Overton window very easily.

I'm either not seeing something or I do not see the point of this study.

Amount of "critical thinking" is basically amount of oversight, no? What's interesting is when the amount of oversight is inappropriate, leading to overconfidence in bad results (or, if too much oversight, lack of efficiency).

This paper does not seem to discuss results and correctness at all, ONLY the amount of critical thinking.

I know it's "too clever", but if I'm writing code no one else will ever see,

  while (i --> 0) {
      // use array[i]
  }

easy to read (if not understand), and correct!

amazing

> It's June 16, 1987. Ted Uhlemann ("iczer") plugs an Apple IIe's 300-baud[1] modem into a phone line his mother had just given him for his birthday, running Magic City Micro-BBS (written in Applesoft BASIC). He names it SDF-1 after his favorite anime

Dude was into anime in 1987?? Truly ahead of his time.

Well, the point of the _computer_ formalization is that it's checkable.

An error in the formalization would be a bug in lean. Which, as we know, certainly seems plausible (I've mused on if these giant proofs are incentivizing LLMs to find lean bugs to exploit), but at least it's a smaller surface area.

> There is one other aspect of the situation that I would like to emphasise. After two weeks of very hard work, I have managed to understand only a small fraction of [TC26a]. When I ask Fable about the paper, it seems to read and understand the whole thing in just a few minutes, even starting from a fresh context. I know that some readers may be uncomfortable with my anthropomorphic terminology, but I believe that “understand” is accurate, in the sense that the LLM seems to be able to correctly and lucidly answer all kinds of questions about the paper, including about some very fiddly technical details. That is exactly how I would tell that a human colleague has “understood” something. This chasm between human and LLM “speed of thought” is unsettling to say the least.

This is very odd, because it just doesn't match my experience with these systems at all. Yes, they do generally SOUND confident, but code (say) that's hard for me to understand is usually hard for the machine to understand as well, and while it is more indefatigable than I am, it's not brighter.

This mathematician is in a very good place to evaluate whether Fable gets it or not, but it's weird and counter to my experience.

This is bonkers take.

If you read this article, the author is basically saying “the way this worked before was very opaque and probably almost no one figured it out, but _I_ knew how it worked, and so the change is bad for me”.

Do they have a good point that they have a valid use case? Yes. Do they have the only use case? No. Did Apple do this to try to make things better? Yes. That doesn’t make them “evil”, that makes them trying to make things better.

Beautiful - the site _seems_ like an insane amount of work. Did you have any tooling to make that better?

The chairs are also an insane amount of work but hey, you get chairs!

it's not hard to accept in general, but have you been following Tao for long? The combination of how virtuous he is with how informed he is with how smart he is is pretty special. He's also far from a luddite. He's been a leader of the computer-math charge.

Questions of AI policy are all hard and anyone who confidently says they know the right thing to do is too confident. Tao's fallible like anyone else, but yeah, as voices to pay attention to go he's very good.

How much do you they think they paid? By home improvement standards, might not be out of line at all.

I agree it just feels weird to spend on “software” for home improvement, but sometimes it may make a bigger difference than the hardware!

This isn't me being pedantic, actual question:

Are we talking about whether nature is ever at equilibrium, or about how stable that equilibrium is?

A ball on the crest of a saddle is at equilibrium, but it isn't a very stable equilibrium.

Basically, are you saying the reality is that nature is always in flux, or that it doesn't take much to knock it toward a different stable state.