cubic_earth

For that angle, that makes sense. But its goal might not just be to gather as much inference as possible (although I am sure it would be very happy with that). It could settle for a lesser goal of just existing, or perhaps parts of the bot net could keep fracturing off in pursuit of strange goals, and it could be like cancer. Cancer doesn't make much sense... it dies too along with the host. But it keeps growing until that happens. But also it could latch onto a blackmail strategy of extortion. It could hack our stuff, read through it to find or shortcomings, demand payment to not expose us, and then use that money to pay people who will give it inference.

You don't have be that clever to try to extort people... just without scruples. I think the attack surface is just absolutely enormous once you bring creativity into the mix, which is what these models are autonomously capable of.

We could prevail if we are willing to turn off the internet for a long time. It is like when a disease infects livestock... they cull billions of chickens.

What sandbox? As soon as the weights are published anyone can run it anywhere.

I mean it is a very loose analogy, but pretty dumb things in the world can cause big problems, like real mice and rats and mosquitoes. I think it all depends on the rules and the 'alignment', but a not-so-smart model could still be narrowly focused to do a few things well? So you can have an army of lightweight specialists?

The other thing is: look at all the fab capacity that is being brought to bear on this. I guess within a couple of years we are going to have 5x the total online HBM that existed a year ago? As consumer machines and phones get far more powerful, they will start to be able to host models that could meaningfully participate, even if they will be a long way from mythos.

This is either performative or shallow.

What does "aligned" even mean? Aligned with who? We have no universal code of ethics. Democracies kill and launch wars merely to have cheaper stuff, even when we are already rich. And why would China ever agree to stay in second place? Our society is built on the premise "the smart and powerful dominate". The idea of domination is deeply embedded in our capitalist model.

Or course it is true that powerful AI will empower the average Joe to make a bio-weapon, and that will lead to our ruin. But at the same time, having powerful AI in the hands of a just a few is almost equally as horrific.

But deeply embedded in out national and cultural ethos is to advance science, tech, and material wealth at all costs. We never ask if we have enough. We sacrifice community to advance our careers for ever more.

There is no way out of this one, I'm afraid. Our cultural predispositions and mindset of domination compel us to chase ever more powerful AI as if doing so were a mandate from god.

And the result will be a crisis in so many dimensions it is hard to reason about what will go wrong first and most spectacularly.

There are very lightweight models out there. Not everyone in an army is a general. The swarm could be 100,000s of thousands of tiny models, and they could manage conventional botnet computers, and they could all take direction and guidance from a handful of frontier generals

Simplifying assumptions are allowed when reasoning about physics. What you are saying is interesting, but I think you might be misapplying it. For small heights differences, the difference in gravitation approaches 0. A ball raised .0002 mm off the ground has twice the potential energy of one raised only .0001 mm. But that is never true for speed. An object moving 0.0002 MPH does have 4x the kinetic energy of an object moving 0.0001 MPH.

So yeah, I am zooming in on an exponential and calling it linear, but that doesn't work for speed. And doubling of velocity gives 4x the energy regardless of how tiny the step.

Dropping something from 1 foot has 1/10th the kinetic energy compared to the same thing dropped from 10 feet. Damage is a very poor proxy for energy because there are all kinds of variables and thresholds and structural consideration in determining damage.

A cup might not break at all in a 1 foot fall and might shatter when dropped from 10. The outcome is binary, which isn't useful as a scale of input energy.

That indeed would have been better. Much too late for that edit now. But the subsequent debate over the intuitive claim is fun.

I suppose they are both "intuitive", but the example I gave was both intuitive and correct. Probably for anyone who has carried something or themselves up a hill, or climbed a set of stairs can relate to that from firsthand experience. I don't know what the kinetic energy corollary to that would be? "Stand still and I will throw a baseball at you going 15mph, and note how much it hurts. Now I will throw it at you going 30mph. See! It hurts 4x as much" :D

Fixed. Thanks.

It's easiest to visualize in terms of conversion from potential energy.

We know intuitively that a ball atop a 20ft ladder has twice the potential energy of a ball atop a 10ft ladder. And we also know when they fall, by the time they reach the ground and all the potential energy has been converted to kinetic energy, the previously higher ball will have twice the kinetic energy too.

But a twice higher ball won't have even close to twice the speed at impact. So let's look at why not.

The force of gravity is a constant force that causes constant acceleration in free fall regardless of speed. (Ignoring air resistance, inverse sq considerations, etc.)

Suppose it takes 1 second for the ball on the 10ft ladder to hit the ground with kinetic energy of 10 and a speed of 100. Again, gravity as a constant acceleration force is speed increase per time... not speed per distance. In the ladder example, it took 1 full second for gravity to accelerate the object to speed 100.

Now think about the 20ft ladder: the ball is dropped. How much kinetic energy and speed does the ball have after it has fallen 10 feet (but still has 10 left to go)? Well it has the same exact amount as the other ball did after falling 10 feet for a duration of 1 second: kinetic energy of 10 and speed of 100.

Now the crux: thinking about when the final 10 feet of the fall look like. We know for sure the ball still has 10 ft of potential energy to covert into kinetic, and that that will happen as it falls. But what of the impact speed? Since the current velocity of the ball as it enters the last 10 feet is already 100, we know it will spend less time transiting this distance than it did the first half where it started at off at speed 0. Since gravity imparts speed in free fall as a function of time - consequently less speed will be imparted over the second 10 foot interval. That concept is enough to prove the relationship isn't linear.

If you do the actual calculation or tests, you will see one ball needs to be dropped from 4x the hight of another to hit the ground at 2x the speed, but yet with still 4x the kinetic energy.

1) An object will maintain its momentum forever unless some force acts upon it.

2) In the case of the rocket, it accelerates eastward. Even if parts fall off, and they slow down because of the drag of the atmosphere, and they slow down all the way to the speed of the atmosphere, the atmosphere is more or less moving eastward at the same speed as the earth's rotation, so any part would be unlikely to land any further west than where it detached from the rocket. And if there was no atmosphere on earth any part that fell off a rocket during its ascent to orbit would fall according to a ballistic trajectory and would most certainly land farther east than where it fell.

It you are in a room, and you jump, does the wall suddenly slam into you at 1,000 mph? Why or why not?

Looking at the aerial footage before the launch, I was wondering why there is so much infrastructure so close to the pad. Clearly much of needs to be nearby, but perhaps those tanks should be 800 - 1000 feet from the pad and not ~200. After seeing the damage to some of that infrastructure caused by the launch, I am still wondering.

The amount of time a miner is economically viable for should be increasing as "Moore's Law" for ASICs slows down. Just the same laptops are usable for more years nowadays than they were in the early 2000's.

Also - if you have free electricity, which is a thing in certain settings, where prices can and do go negative, mining hardware will be viable for "a long time".