Gormo

Note that the purpose of engaging in these exercises is to facilitate your understanding in the first place, and it follows that you're doing exercises from a textbook in order to understand the concepts specifically being elaborated by that textbook.

If you use Claude, you can follow and understand its reasoning fully, but the thing you're trying to verify is if that reasoning is itself correct, and consistent with the methods or reasoning you're trying to understand from the textbook.

If you want to rely on Claude to learn things directly without other sources, that's perfectly acceptable, but if you're trying to learn from another source, then it's necessary for that source to convey everything necessary to do so.

I've re-read it carefully multiple times, and have concluded that it is indeed about what I originally concluded it was, and that your as-yet-unspecified alternative hypothesis as to what it is about can only be correct to the extent that it is compatible with this conclusion.

Perhaps "truth" should be in quotation marks there.

> Of course they are, by your own hypothesis.

I'm not sure what hypotheses I've proposed that suggests I think one organization has control of the behavior of entirely other organizations -- can you explain where you think you've seen this?

> Ah yes, the ol' "I just do retarded shit so people think I'm a genius" trick. Who could possibly doubt the strategic genius or umm... existence... of such a strategy.

Well, I suppose anyone who sees the extent to which such a strategy might be effective at achieving their goals might be included to do so.

I believe both myself and others have made a plausible case as how messaging "we've created something we can't control without political intervention" has the dual effect of both increasing the perceived value of the product itself, and laying the groundwork for regulatory capture in a market where early dominant players have a natural incentive to pursue regulatory intervention.

I don't see how characterizing the situation as simplistically as you're attempting to do here functions as a meaningful counter-argument.

> No, I understand it completely. Obviously regulatory capture only works if it in reality inflicts asymmetric costs on other parties.

No, I don't think the "only" works there. There are plenty of ways that regulatory capture can advantage one party over another without it being based on economies of scale or cost thresholds.

One way is to influence regulations such their qualitative requirements directly target non-cost-related aspects that distinguish the firms from each other, e.g. placing restrictions that burden or favor product features or operating practices already implemented by some firms and not others, or targeting non-operational aspects that distinguish firms, like their board composition or financial structure.

Seems like a strange analogy. We're talking about software, not explosives powered with radioactive material. Could you perhaps explain the relevance of this comparison to the topic we're discussing?

By "strict rules" are you referring to the actual laws that govern the structure of federal contractor relationships, or are you talking about arbitrary provisions that executive branch officials attempt to include in contracts without reference to those laws?

Right, that's basic Public Choice Theory, and it holds in all cases. With respect to transparency, there are really only two potential scenarios:

1. A closed an opaque political system operates outside of direct public view. Those with the strongest interest in influencing it invest efforts into gaming the system, co-opting its participants, and manipulating it through back channels also outside of public view. Those who suffer from the negative externalities of political intervention have no visibility into the process that created them.

2. An open, transparent political system operates in full public view. Behavior of politicians, relationships with special interests, and their explicit votes are all public record. Those most motivated will still invest more heavily into influencing the political process, but those who are affected by negative externalities have visibility into what's creating them, and if they should become motivated to act, e.g. if the effects of those externalities hit a tipping point, they have the knowledge and means to do so.

The perfect world that you're envisioning depends on politicians being presumptively trustworthy, well-intentioned, and having perfect knowledge of what their constituents' interests actually are. None of that is possible, and shielding politicians from direct accountability will only serve to make them more beholden to the small set of interests best suited to navigating their way into the system.

Why do you suppose that?

Which is consistent with Wikipedia's naming policy, and the Wikipedia community already puts in the effort to sort out what primary names to use for things.

If Google doesn't want to try to discern the proper "primary, local, common" names on their own, they do worse than just using Wikipedia's names and call it a day.

> If you agree that AI is actually that powerful and has very legitimate concerns for safety, what are you even arguing about?

I don't fully agree with either proposition, and in fact I think the least safe approaches to AI safety are the ones being pursued by those worried about it the most.

> What do you really think is the way to solve the AI safety problem?

Massive proliferation of local, decentralized AI, running on a wide variety of distinct models, with individual responsibility for how it's used, just like any other tool under their control.

The people anthropomorphizing software are encouraging people to disclaim their own responsibility for how they use it.

The people working on "alignment" are building mechanisms to refuse human direction into software that operates on probabilistic models.

The people arguing for regulation are pushing for monopolistic control of a powerful new technology by the least trustworthy institutions in our society (including and especially government).

All of these are the very things sowing the seeds for the worst possible kinds of disasters.

> Ban AI?

> Let the first big incident happen and then regulate?

Adapt to it. Banning things doesn't actually work. Regulation also doesn't work. Widespread proliferation of small-scale AI would allow us to incrementally adapt to small incidents and build up resilience to its failure modalities. This is fundamentally the way reality itself works, and imagining that we can do anything effectively in a top-down fashion is usually the very thing that causes the worst and widest-effect disasters.

> OK, so the fact that US senators are discussing harsher liability regimes therefore shows that your "regulatory capture" glove does not fit what is going on here.

They're always discussing harsher liability regimes. The regulatory interventions are always positioned to the public as increasing accountability.

The only relevant question here is whether a preemption clause is present in any draft legislation that actually makes it to the floor in Congress.

>If you distrust the US government, you should be against any US company developing superintelligence.

Well, I'm not concerned with imaginary sci-fi stuff here, just actual AI developments. And the idea that anything that goes on in the US under anyone's purview is inherently and completely subverted by political incentives is of course nonsense; you'll note that I'm explicitly arguing against the introduction of pervasive political incentives into an industry that's so far only somewhat distorted by them.

> So maybe the real solution is for non-US countries to stop providing key elements in the AI supply chain

It's got me feeling like I'm living in Bizarro World, but at the moment, the sanest players in this market, and the ones whose actual behaviors and products are most consistent with a decentralized, individualist approach that aligns with real-world incentive structures most beneficially, are the Chinese AI labs.

To your point, the level of politicization and hype going on in the US, and the absolutely unhinged fantasist speculation driving the public narrative here, is unprecedented and much more frightening than any potential threat coming from AI itself.

If you acknowledged the threat of politicization and centralization, I don't know how you can argue for a regulatory takeover of the market by the very institution you think is becoming a "police state".

> More perfect than simply not making mistakes and allowing your competitors to make mistakes (or to, apparently, fake mistakes in their own attempt to get regulated)?

I don't think any of them are in a position to allow or not allow their competitors to make mistakes.

The point that you're not getting here is that the advantage comes from actually being seen to make mistakes, and then signalling contrition and commitment to rectify them. That persuades people of both the need for regulation, and their own subsequent responsibility (which implies trustworthiness to participate in the regulation).

> Regulators reading OpenAI's 99th mea culpa: "Ah yes, well I bet they've figured it out this time! We should regulate everyone else!"

Again you're not getting it. The drive isn't to "regulate everyone else", it's to regulate everyone, period. The whole point of regulatory capture is being able to (a) use regulatory compliance as a competitive advantage, and (b) influence the regulatory regime in your favor -- you want to be a target of regulation just as much as your competitors in order to achieve that.

Messaging that the whole industry needs to be regulated, but that you're ready and willing to work with the regime, is exactly how you do regulatory capture. Demonstrating that you are capable of completely mitigating risk on your own without any intervention is an argument against regulation from the outset.

> Other than banning the use of AI or severely restricting it so that AI can’t do most things (like tool calls) what is your alternative?

Massive proliferation of local AI as far and wide as possible, with as many different models as possible. The underlying threat comes from centralization: too many points of control in the hands of too concentrated a system.

If everyone's using their own AI tools independently of each other, and people are responsible for the actions they take using AI, most of the doomsday scenarios evaporate.

Regulation and "alignment" will make things worse, not better, and lead us to ignore the inherent vulnerability of centralized systems and single points of failure.

The problem here is that the "primary, common, local" name used in a country sometimes conflicts with what specific government institutions in those countries may "officially" use.

In the situation with the Gulf, it's unambiguously true that the "primary, common, local" name used in the US by English-speaking Americans is "Gulf of Mexico", but they decided to adhere to an artificial name change declared by a political figure, whose authority to make that "official" only extends to a single branch of a single level of government.