joshjob42

I'm a physicist and there is approximately zero chance no one uses AI in two years unless we're all dead. It's a world class mathematician and expert software developer with the capacity to research papers 100x faster than I can, which has given me (like most scientists, an okay programmer at best) to be able to test vastly more ideas and find useful papers etc far faster than I ever could before.

I'm guessing you don't have any tasks in your life that involve coding, math, science, or law if this is your view.

Also literally the post he links to for "its bad actually" is a blog about how it's great and not very addictive and just has some tweets from people who apparently didn't pay attention to the nicotine gum/patch/pouch recommendation in gwern's post and instead started using various forms of tobacco. Very sloppy all around.

What makes publishing it valuable? At best it's a second round of PR for the paper, after it was initially released on the arXiv. Reviewers sometimes have good suggestions, but you can get suggestions from people on your arXiv paper and post a v2.

From the perspective of advancing science, journals seem mostly pointless and most of their benefit could be had at next to no cost if we just agreed to publish on the arXiv and pair it with a review system.

From the perspective of advancing scientific careers, because of how founders and institutions treat arXiv papers, you have to still publish in a journal, and so people keep publishing in journals.

But like, at least in physics time passes at the speed of arXiv posts, and getting published usually happens after it's been cited many times already. And if people are already incorporating it into the web of knowledge/responding to it etc., then what did the journal do?

I mean you can never be sure your workers aren't just making shit up with confidence. Like, the humans who work for you. And yet somehow, we've built all this software.

How to produce systems that work reliably out of a collaboration of large group with one person at the head who ultimately has responsibility is more or less the study of most human endeavors, this does not seem like an insurmountable problem.

Kinda sounds like the problem there is that llama.cpp has decided it doesn't want any of the ideas of its users that they build on top of it. If I built a feature that solves my problem, I'm not going to completely redo it from scratch by hand just to make a PR. There's a lot of stuff that would be valuable to build that doesn't because people who have the idea don't have the time/energy/etc to pursue it. Now they have an assistant they can delegate to and have the energy, but only because of the assistant being able to take on large portion of the work in the area they're not as expert in.

So if you reject projects largely coded by supervised AI, then you simply won't get any of the value from them.

In a sense, all these features and ideas etc were always things that could have existed, but before they sat in the fog of the invisible, and now you get to see many more of them and it can be frustrating. But in a no-LLM world we weren't going to get those features and products anyway.

If people want to build stuff they can, they just may not be paid for it. But if we don't need to pay anyone to do anything then it also doesn't matter that you aren't being paid, as you'd have all you need, and you can spend your time tinkering away on low tech/cottagecore enterprises like designing an ASIC or making bread.

Why does their lifestyle require p(doom) to be large? Because they work at orgs focused on AI safety? I'm not sure what the claim is here.

Maybe, but I suspect that a lot of the actual things that produce the algorithm, proof, etc are not actually explicit reasoning. Q: "Why did you try that vs some other thing at this point?" A:"seemed right to do"/"kinda looks like other stuff I've seen before"/"first thing that came to mind" etc.

Why pursue this one avenue vs some other, typically its cause it seemed more promising, and the person can come up with reasons but did they explicitly verbalize a fully sound chain of thinking at the time? Probably not.

Not to say that the explicit thinking, or writing things down, isn't important. But if I examine the process by which I develop a proof or something, there's a lot of vague hunches, blind alleys, etc that come along the way. And, many of the blind alleys etc probably aren't actually that important in the end for me finding the right answer -- if you were to trim that part out of my own internal reasoning trace but left the rest intact, I'd still get the right answer because, well, it was a blind alley.

Seems like the obvious solution is to put a bank of floor numbers above each elevator and when one arrives the floors it has been assigned to go to light up and the people going to them get in that one. So each person only needs to quickly to see if it's going to their floor or not, much like they need to look to see if it's going up or down. Then you can dynamically dispatch elevators right up until 5s or so before it arrives at a floor, with full knowledge of not just up/down but also destination. (And hell make 'em say how many people are in their party too so you also know how many people are traveling and where.)

Sure and if she was funding high cost high risk high reward research I'd say that's a good case for that being higher in expected value but higher variance than GiveWell, but she is mostly not doing that, instead giving to basically normal causes and institutions to spend money on the things they were anyway.

The value of a billionaire lies on their ability to concentrate large sums of capital on high risk projects with enormous upside. Spending money instead on fairly pedestrian applications of money for philanthropy doesn't do that, she has more closely approximated taking an index fund or generic philanthropy. That's fine but has none of the benefits of concentrated wealth to kick us over local maximuma and none of the enormously higher utility of genuinely effective causes.

It's a real shame that she has wasted her money so catastrophically. You can save a life via GiveWell for what 5k or so? At best she saves one year of life for a dozen times that. On a QALY basis, she's probably 300x or more worse. I'm not saying someone has to give all their charitable giving to GiveWell or similar, but if you're not you should probably have a very strong argument why not (for instance, high risk basic research you find unusually promising, or trying to solve an engineering challenge that would unlock huge value etc), or admit to yourself that you aren't primarily trying to help people but to further some particular value to yourself. You could donate $5k to the local school so they can renovate the art room, but in doing so you're deciding to do that rather than save a life. That's fine, but it's functionally more like buying concert tickets or a new TV -- the improvements you're making in others lives are tiny by comparison to what you could have bought with that money instead (a save life with GiveWell), so that part obviously has negligible weight in your decision, it's primarily about your personal enjoyment of furthering particular values. Again, that's fine, we all do that. She's just decided to take a course of action that is much closer to buying giant yachts and football teams or a vanity run for office than try to make the world a better place while trying to look like she's trying to make the world a better place.

I mean do a bunch of simulation work and statistics stuff but now jack about website design, so if I ever make a website about any project of mine it'll be vibedcoded or a barebones export of an org file README on github. Websites have little bearing on programming language runtime development, no?

Or that GNU updates a policy that will very rapidly go from probably net silly/mildly contestedly useful to completely ridiculous in a year or two. Not allowing LLM code will be basically turning down the work of the worlds best programmers running at 50x speed in couple years, and will functionally doom any software project that enforces such a policy.

Add LuaLaTeX and you're cookin' with gas. For real would be fantastic if we could get more or less the full LaTeX ecosystem readily and rapidly available online and in a huge variety of desktop applications.