snaking0776

Why is 2 a direction to be avoided? Is it really so bad to prevent an individual from having majority voting rights over a corporation which has power on par with most nation states? I’m not sure a billion dollar cap is the answer here but surely there must be some kind of reasonable ceiling. Is it so bad to think that one individual shouldn’t have majority ownership over a company dominant for the US space program? Or control over major international internet access that’s useful during conflicts? Or major social media platforms which provide algorithmic dials to tweak what information is visible?

I think there’s really only a few solutions to it which all involve some kind of divestiture, it just depends on if you want to target corporations or individuals:

1) a new wave of anti-trust ala Lina Khan and breaking up most major tech companies.

2) passing laws limiting the behavior of, and allowing for the punishment of corporations and group agents as a whole. There’s a lot of philosophical work about how to hold a group accountable for their actions, try searching List and Pettit Group Agency. There’s been a lot of development since then but I’m less aware of it.

3. Caps on decision making power (potentially measured via some weighted stock ownership, or wealth more generally) like suggested above. I agree there would be a lot of second order effects here which are beyond the scope of this comment but I think we’ve clearly gone too far in one direction where single individuals are now essentially funding national party midterms, have increasing power over real world conflicts, and any other of the myriad of corruption issues you can find out in the wild.

Two things can be true. Open AI is a huge company.

1. It has people in it who are career obsessed and who are willing to ruthlessly go after any opportunity to improve their standing/stock valuation. Using a 2 week old model to snipe a millenium prize for PR is in line with that.

2. There are many researchers and even executives at the company who genuinely think we are speedrunning the end of the world. I don’t know anyone in this field who honestly argues that if we build ASI soon it doesn’t lead to extinction. This group of people can output warnings about the state of research and fears for the future while pushing for regulation out of genuine fear of what they’re building. I tend to agree with them.

You can’t view these companies as a monolith. They’re actions won’t be consistent because it’s built of many people with conflicting beliefs. Please look at arguments regarding AI risk and the current pace of progress and value them as it relates to the argument itself, not who said it. We are in a dangerous place and no one is sure how quickly we’ll get to a bad spot.

Fair, I was going off memory and was wrong. I guess I was mostly just trying to get across that communication seems to be more powerful than we give it credit for, but should’ve double checked.

I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.

Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as:

1. Robotics begin rolling out more broadly across the world.

2. Labs start automating more and more of the physical process of running science as expectations of natural science advances begin to mount.

3. Economic pressure between the labs continues to ramp up and the pressure to continuously improve forces quicker and quicker model releases than a team of human scientists can effectively evaluate outside of automated means.

No one knows what pre-conditions are for us to hit the point of no return nor how quickly it will come. If all is required is a sufficiently advanced cyber model we may not be far off. If it requires incredibly complex biological knowledge and access to certain lab supplies we likely have a bit longer. Yes this is guess work and we need more evidence of the dangers but at the same time we need evidence of safety. While you may disagree with the risk level, I think it is easy to see the consequence if these labs achieve their stated goal. At this point it seems a political solution is the only way to enforce caution.

While it would be insane to get a PhD if you had to pay for it, at least in the US you are paid a stipend (if not a good one) so you can generally avoid any debt. There is definitely an opportunity cost though.

I hope this is sarcasm. I sometimes can’t tell here

I’m not so sure this is trivial and strongly depends on the biases of the agents that the group is composed of. It seems like some species of animals (humans, ants, wasps, etc) seem to select for intelligence in group settings more than individual intelligence. One of my favorite facts about human evolution is that in the process of becoming homo sapiens our brains actually shrunk relative to our ancestors possibly due to our reliance on social ties and specialization allowing for each individual unit to be “dumber” while the collective becomes more intelligent than any individual ever was.

It’s not apparent to me that current LLM’s have the right biases for a trivial unit of “more LLM” = smarter. It seems apparent that LLM’s are under many metrics more “intelligent” than the average person but they still don’t rival our corporations or collectives in terms of intelligence and I don’t see them being plug in replacements for humans and maintaining the benefits of the collective structures yet. Not impossible but definitely not trivial in my mind. We and our cultures evolved to work together and the modern world is the result of the emergent structure that resulted.

As a plug my favorite field relating to this is called stigmergy and basically describes how individual units like ants build intelligent collectives that are far greater than the sum of their parts.

Depends on who you’re talking about. How does the cost benefit analysis work out for someone while lives within a mile of a datacenter, owns no shares in the company, and pays rate fees for their utilities and will have to pay for maintenance for the increased load capacity as well as higher spot prices? If you view America as a cohesive unit I’d agree with you, but I don’t think these people are necessarily acting irrationally if you view their personal circumstance. It’s up to the building company to sweeten the deal and provide an incentive for the locals to add it. Based on the contracts I’ve read from many of these datacenters it seems more like they’re buying the pockets of corrupt politicians and developers in areas with high political inequality while conferring very little benefit to the people who deal with the externalized cost.

While I’m sure there are people profiting off these movements, there is very real grassroots support for blocking datacenters and generally anything relating to AI and I can’t blame anyone other than the companies building it. I think in 30 years this will be studied as the worst marketed technology ever. It’s one thing if outsiders try to say your technology is dangerous, it’s another if the companies themselves are telling the public that they’re technology is either going to a) exterminate humanity or b) eliminate all jobs and institute a permanent underclass. I think they’re right that these are concerns and while our niche tech bubble maybe thinking also about the potential scientific benefits of the technology, all the wider world sees is multi-national tech companies taking advantage of areas with high political inequality to push through genuinely unfair contracts that saddle utility rate payers with higher fees, potentially negative health effects, and with very little long term job creation. This is also coming from companies that have ignored complaints about the overriding addiction and division which their platforms have allowed to propagate over the last 20 years all while many Americans watch historic levels of corruption in government in favor of those same companies go unpunished. At this point I think the tech world has really burnt all of its political capital with the wider populace and while we may sit back and complain that progress is being blocked by luddites, its not irrational to want to block the creation of a technology which you will pay the cost for while seeing a disproportionately small percentage of the rewards. I think if we’re going to see the actual proliferation of this tech and the benefits it will bring, their needs to be a) more publicized news of real benefits to human quality of life in visible fields like medicine and b) an actual plan for how to confer the rewards of technology more widely rather than just continuing to center in the hands of so few people and not just approaching it as higher gdp but also lower wealth inequality as long as its correlated with political power

I like that rule. I’ve also found that for most things that are risky enough to warrant a second opinion it will probably take long enough to get in that you can schedule it several months in advance and then use the intervening time to learn more although calling medical offices to schedule appointments is no fun.