evenhash

Agents don’t have infinite working memory…

LLMs benefit from abstractions for the same reasons that humans do. More information in the same amount of text. Fewer working parts to juggle so fewer ways to make mistakes.

What I’m saying is that you can’t just trust what the AI says because it produces a Lean artifact.

The statement of the theorem has to be correctly translated from English into Lean code.

It’s like translating user requirements into code. The code could run without bugs but not do what the users want.

The only way to know the AI did it correctly is to check. You can’t just take it at face value.

"Lean-verified" is not some magical incantation that makes a supposed proof irrefutable. Even disregarding potential bugs in the kernel as others have said.

Say that AI gives you a Lean proof and says it proves Theorem X. It could just as easily give you the same proof but claim that it proves (not X). How would you know the difference?

Nothing can really be considered proven unless a human expert can read the Lean proof and determine that (X as defined in the Lean proof) corresponds to X. The proof (at least the statement of the theorem) must be intelligible to humans to have value.

It's possible people will just start taking AI at its word. Maybe AI says "Here is a Lean proof of X" and we all just shrug and go "Okay, X is proven." But that's not how it works right now for human mathematicians. Why would we apply that standard for AI?

I think that would only follow if the rest of the world’s purchasing power also grows at the same rate (unlikely), or if you were to cease all international trade (uhh…).

> An AI-generated solution always provides ... proof that there is a solution

This is only true in the most trivial sense. A solution is a solution, sure... but how do you know it's a solution, and not an incoherent jumble of words? A human has to review and vouch for it.

Just because the AI gives you an arxiv-worthy PDF, or a Lean proof which compiles, doesn't mean it proves what the AI says it does. The AI could give you the same PDF/Lean code and says it proves the opposite, how would anyone know the difference?

You can't advance human understanding unless you produce things that humans can understand.

This is already a thing? Just replace “robots” with “machines”.

And yet I see videos all the time of Indians running contraptions straight out of the steam age in their shacks to make plastic straws or sandals. Or just doing it by hand. Better machines to do these tasks already exist, but are obviously too expensive for them to afford. Why would these hypothetical robots be any more affordable or disruptive to Indian workers in particular?

If anything I think American blue collars would be the ones to worry about.

> It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal.

No big deal? It’s not like it’s free… tokens cost money.

Sort of. For me it’s more like seeing out of a third eye (while also seeing out of your real eyes). Like how if you put your finger right in front of your face, you can see two images of it at once. With concentration I can ignore what my real eyes are seeing to prioritize the visualization, but the real eyes are constantly pulling my concentration back.

One thing you notice running NoScript is how prevalent Google is.

Even if a site doesn’t monetize with Google Ads, there’s a decent chance it’s pulling a script from ajax.googleapis.com, and Google still knows that you visited the site from the Referer of the script download.

If what you’re saying is true, what’s the use (or even meaning) of being “all in”? You’re a rock either way.

This is where I’m at as well. If it needs to change too much to do what I want, I take that as a sign to either (1) ask for a change that is easier to review, or barring that (2) pivot to refactoring the codebase until it becomes a change that is easy to review.

Ironically, even in this era of cheap and instant code, what works best (for me) is still to write as little code as possible.

Being “verified in Lean” doesn’t magically solve the problem of hallucinations unfortunately.

It just shifts the work from

> reading the (natural language) proof and confirming it has no errors

to

> reading the Lean code and confirming it correctly encodes the theorem

For example here is a statement of the Pythagorean theorem in Lean:

theorem EuclideanGeometry.dist_sq_eq_dist_sq_add_dist_sq_iff_angle_eq_pi_div_two {V : Type u_1} {P : Type u_2} [NormedAddCommGroup V] [InnerProductSpace ℝ V] [MetricSpace P] [NormedAddTorsor V P] (p₁ p₂ p₃ : P) : dist p₁ p₃ * dist p₁ p₃ = dist p₁ p₂ * dist p₁ p₂ + dist p₃ p₂ * dist p₃ p₂ <-> angle p₁ p₂ p₃ = Real.pi / 2

This is just one possible way of formalizing it and it depends on other definitions, wherein you also need to understand the assumptions they make, etc.

Answering the question of “whether proving this theorem in Lean proves the Pythagorean theorem” thus requires expert judgement as well as domain knowledge of Lean’s libraries.

So if the AI says “this theorem is true, here is the proof in Lean” it’s still possible that it’s not correct, even if the Lean code compiles. The result will still be in question until a human expert reviews it.

I’ve found it depends on where I connect from. If I’m using my home WiFi, it works. If I use 4G or a VPN, I get the login prompt. Still seems to work on my WiFi at the moment.

It’s not crazy when you consider that their customers are the advertisers, not you. They want to keep their customers happy and buying more ads.

Not everyone works for Evil Corp. I work in the public sector and my work supports public health and safety initiatives. AI has allowed my team to get much more done than we would have otherwise which improves the quality of life of the people in my community.

So I would like to counter your cynicism with a “YMMV” depending on who you work for.