PollardsRho

What about bets without insider participation, where you want the market to function as an aggregator of educated guesses? OP has one reaction to insider trading, but I imagine a very common alternative would be "those insiders make their money off of bettors like me, I shouldn't participate." Some questions are clearly insider-proof, but I imagine many questions have insiders who don't bet on Polymarket. If Polymarket is going to be a good prediction market, surely it should incentivize people to make predictions on those questions too?

A lot of expressions in English started out as calques, outputs that process: you're paving the way!

All models are wrong; some are useful. Cognizance of that is even more critical for a model like exponential growth that often leads to extremely poor predictions quickly if uncritically extrapolated.

I think "are the failures of a simple linear regression on the METR graph relevant" is a much better framing than "does seeing a line if you squint extrapolate forever." As I said, I'd much rather frame the discussion around the actual material conditions of AI progress, but if you are going to be drawing lines I'd at least want to start by acknowledging that no such model will be perfect.

Students shouldn't be treating class material as something they "do not care to know."

AI can be used in ways that lead to deeper understanding. If a student wants AI to give them practice problems, or essay feedback, or a different explanation of something that they struggle with, all of those methods of learning should translate to actual knowledge that can be the foundation of future learning or work and can be evaluated without access to AI.

That actual knowledge is really important. Literacy and numeracy are not the same thing as mental arithmetic. Someone who can't read literature in their field (whether that's a Nature paper or a business proposal or a marketing tweet) shouldn't rely on AI to think for them, and certainly universities shouldn't be encouraging that and endorsing it through a degree.

I think the most important thing about that kind of deeper knowledge is that it's "frictional", as the original essay says. The highest-rated professors aren't necessarily the ones I've learned the most from, because deep learning is hard and exhausting. Students, by definition, don't know what's important and what isn't. If someone has done that intellectual labor and then finds AI works well enough, great. But that's a far cry from being reliant on AI output and incapable of understanding its limitations.

If you have many metrics that could possibly be construed as "this was what we were trying to improve", that's many different possibilities for random variation to give you a false positive. If you're explicit at the start of an experiment that you're considering only a single metric a success, it turns any other results you get into "hmm, this is an interesting pattern that merits further exploration" and not "this is a significant result that confirms whatever I thought at the beginning."

It's basically a variation on the multiple comparisons, but sneakier: it's easy to spend an hour going through data and, over that time, test dozens of different hypotheses. At that point, whatever p-value you'd compute for a single comparison isn't relevant, because after that many comparisons you'd expect at least one to have uncorrected p = 0.05 by random chance.

Advantage, sure. I just don't think that advantage is particularly meaningful in situations a human has virtually no chance of escaping. Humans also have a lot of their own advantages. How is a chatbot supposed to cross an air gap unless you assume it has what I consider unrealistic levels of persuasion?

I think you also have to consider that AI with superpowers is not going to materialize overnight. If superintelligent AI is on the horizon, the first such AI will be comparable to very capable humans (who do not have the ability to talk their way into nuclear launch codes or out of decades-long prison sentences at will). Energy costs will still be tremendous, and just keeping the system going will require enormous levels of human cooperation. The world will change a lot in that kind of scenario, and I don't know how reasonable it is to claim anything more than the observation of potential risks in a world so different from the one we know.

Is it possible that search ends up doing as much for persuasion as it does for chess, superintelligent AI happens relatively soon, and it doesn't have prohibitive energy costs such that escape is a realistic scenario? I suppose? Is any of that obvious or even likely? I wouldn't say so.

If someone who is so good at manipulation their life is adapted into a movie still ends up serving decades behind bars, isn't that actually a pretty good indication that maxing out Speech doesn't give you superpowers?

AI that's as good as a persuasive human at persuasion is clearly impactful, but I certainly don't see it as self-evident that you can just keep drawing the line out until you end up with 200 IQ AI that is so easily able to manipulate the environment it's not worth elaborating how exactly a chatbot is supposed to manipulate the world through extremely limited interfaces with the outside world.

I don't think there's a confident upper bound. I just don't see why it's self-evident that the upper bound is beyond anything we've ever seen in human history.

People are hurt by animals all the time: do you think having a higher IQ than a grizzly bear means you have nothing to fear from one?

I certainly think it's possible to imagine that an AI that says the exactly correct thing in any situation would be much more persuasive than any human. (Is that actually possible given the limitations of hardware and information? Probably not, but it's at least not on its face impossible.) Where I think most of these arguments break down is the automatic "superintelligence = superpowers" analogy.

For every genius who became a world-famous scientist, there are ten who died in poverty or war. Intelligence doesn't correlate with the ability to actually impact our world as strongly as people would like to think, so I don't think it's reasonable to extrapolate that outwards to a kind of intelligence we've never seen before.

Why is 2) "self-evident"? Do you think it's a given that, in any situation, there's something you could say that would manipulate humans to get what you want? If you were smart enough, do you think you could talk your way out of prison?

Thanks for sharing this proof! As someone who enjoys math but never got myself through enough Galois theory to finish the standard proof, it's fantastic to see a proof that's more elementary while still giving a sense of why the group structure is important.

At that point, you'd be better off just using a recursive algorithm like the one in GMP. You're swapping out arbitrary-length for arbitrary-precision.

The compound-interest intro to e (the value of 1 dollar compounded continuously for a year at 100% interest), to me, has several useful advantages over different introductions that are more mathematically rich:

- It's elementary to the point that you can introduce it whenever you want.

- It automatically gives a sense of scale: larger than 2, but not by a lot.

- At least to me, it confers some sense of importance. You can get the sense that this number e has some deep connection to infinity and infinitesimal change and deserves further study even if you haven't seen calculus before.

- It directly suggests a way of calculating e, which "the base of the exponential function with derivative equal to itself" doesn't suggest as cleanly.

I don't know of any calculus course that relies on this definition for much: that's not its purpose. The goal is just to give students a fairly natural introduction to the constant before you show that e^x and ln x have their own unique properties that will be more useful for further manipulation.

I will die on the hill that TOML should be used for the vast majority of what YAML's used for today. There are times a full language is needed, but I've seen so many YAML files that use none of the features YAML has with all of the footguns.