remus

I think you are looking at wealth vs. income?

I think they're just writing cautiously because they're midway through the investigation and don't want to give false confidence. Doesn't seem worth reading too much into it until they provide a more thorough write up.

> I don't want it to do "gradual typing", if it is unsure about what type something is, I want that to be a failure instead of it modeling the unsureness and not showing potential errors. We live in the age of agents, having to add an extra annotation here and there is not so much of a problem any more.

A good goal, but a hard sell for a tool when the underlying typing spec is very flexible and designed to allow for gradual typing. And of course there are all sorts of python codebases with everything from no types to very tightly specified types.

It's not quite as egregious as someone purposely choosing the key. According to the write up on the Mozilla bugtracker there was a bug in the script used for there key generation ceremony which caused the bad certificates to be generated. Obviously completely unacceptable for a CA, but makes more sense than someone plucking 4 out of the air while generating a root cert.

I think transparency in government is really important, but I don't think making ~all communication between officials public is the way. For the well behaved people it has all sorts of funny second-order effects, like people being reluctant to ask questions which might make them look silly in case they're taken out of context. For the poorly behaved they'll just work around it.

While I think there is some truth to this I think the framing is very uncharitable. The obscurity in maths comes from layers upon layers of abstraction because this is a useful way for humans to work with these complex ideas. I think we see some of this with these proofs that have been translated to lean: some of the proofs are millions of lines of lean, imagine trying to wrap your head around a codebase that is millions of lines! And just for a single proof! These impenetrable abstractions that mathematicians have created are like a mental model for a massive code base.

I think the question is then "how do I gain more confidence this code is correct?" This doesn't have to be reading through everything line by line, it can be writing more tests, or working with the LLM to dig into how certain edge cases are handled etc. There are a lot of things you can do to improve your confidence in a piece of code.

> Tesla's stock price might be detached from reality, but is their actual business in trouble?

I am not sure you can detach these two concepts. Tesla is currently valued around 3x the next top 5 auto companies combined, so to an extent you need to judge the company on these expectations. I happen to think they do have a pretty good business selling cars, and maybe a very good business selling autonomous cars, but there's so much uncertainty I think it is hard to see the current market cap as anything other than hugely inflated.

> It's not a tradeoff, it's poor design.

How so? There seems a fairly inherent tension between having things just work by default and hiding surface area for fingerprinting behind permissions pop ups. For example say you go to a maps website, how do you decide whether to share the users location with the website? Or enable webGL functionality to render some fancy layout on the screen? As a user having to enable all that stuff per-website is annoying.

Maybe there's some other way Im missing?

It's a UX trade off. For non-adversarial websites it's very convenient for users because things just work without having to go through loads of permissions pop ups for every website, but of course this then makes this info a juicy target if you're trying to fingerprint someone.

I don't think this really works out. Putting a data center in orbit is basically a question of moving lots of mass which is always going to have a large energy cost for accelerating all that mass. On the other hand someone wanting to damage that data center only has to get the mass of one decently sized bomb to the same velocity.

Unless you're talking about an adversary who has ~no launch capability then the person trying to put the data center out of reach is always going to be massively disadvantaged in terms of the physics.

> In other words I want to spend 100% of my mental capacity in the problem domain for the things AI cannot do for me, like steering, grounding, verification and not for things AI could do.

It sounds like the parent is less talking about this, and more people burning tokens while not getting useful work done.

> Similarly now we're getting AI doing math. The proofs compile but are a mess. So just make the models better at writing clean proofs and explaining what they're doing to humans. That's the end of it.

The assumption here is that the true/false of the theorem is the important outcome. While it is certainly part of it, a big part of maths is the understanding you gain from a proof. Many of the best proofs elegantly explain some aspect of the maths which was previously unclear and expand our understanding of the world.

To use a programming related example, imagine that an LLM spits out a solution to the travelling salesman problem which works in O(n) time. On the one hand that's very convenient for whatever problem you happen to be trying to solve at the time...but there's also an answer to P=NP in there! The former means your delivery drivers app works a bit faster on their busy days, the latter fundamentally shifts how humanity thinks about certain problems.

Going back to the maths, there have been theorems that were proved (by people) where the proof is broadly seen as 'unsatisfactory' in that it doesn't really expand our understanding. I assume some of these LLM proofs are a bit like that: we now know that the thing is true, but we really want to know why it's true, and how that changes our understanding.

personally I do not take such a hard line, I think a more limited form of copyright does have a place in allowing authors to profit from their work.

I think what doesn't work is a massive, blanket term applied across all works. The situation we have currently is a handful of extremely valuable works which get milked for decades and decades, and huge bulk of material which is left to rot into obscurity because it is untouchable under current copyright law.

I have experience of this with a project I run collating historic material on climbing and mountaineering: pre-internet a lot of the discourse happened in magazines, and this discourse is generally pretty hard to get your hands on. You need to track down physical copies of obscures mags that have been mouldering away in someone's attic for the last 40 years. If you talk to the authors of the content in these magazines, none of them are bothered about trying to make money of 500 words of copy they wrote 40 years ago, but because of the current massive copyright terms this stuff will not become public domain for 80-90 years in many cases, by which time I am sure finding physical copies of this material will be very difficult.

I would like to see a much more limited term by default (20 years from publication perhaps?) which still gives an author plenty of time to capitalise on their work. I think it should then be possible to renew your copyright claim (every 10 years say?) but at an increasingly high cost. £250 the first time, £500 second time, £1000 third time etc. This would mean the bulk of material would fall out of copyright on a much shorter timescale, while if you happen to have made something popular and you want to keep benefiting then you can up to a point.

How would they know for sure that some details were not part of some other training data they use? The authors may have discussed some tangential details on a forum for example, in which case you might argue that the model picked up on these details the authors assumed were benign but novel and worked out how to apply them to the problem.