sometimelurker
Joined 6 months ago
Karma
126
Submissions
208
Member since
May 15
Recent submissions
Comment · 4 months ago
> Where in nature has anything ever grown exponentially in perpetuity?
you could make the argument for human total GDP, which looks like https://ourworldindata.org/grapher/global-gdp-over-the-long-..., but then again, you could say we just haven't reached the sigmoid in it. I personally doubt we will
Comment · 4 months ago
LLM + scale = more intelligent, this can be proven more than empirically, https://arxiv.org/abs/2409.15318 shows that neural nets can fit a number of independent AND-gate operations in their weights,
- if you have a system that is large enough to store, lets say 10^12 AND gates (all frontier llms can do this) - and this system can produce outputs based on previous things it has outputed
its turning complete, and RLVR on it is optimization over the space of algorithms. If an algorithm exists to do a task, and the task can be verifiably done, this finds the algorithm to solve the task most often.
it is obvious that this scales, from much-worse-than-human to slightly-worse-than-human, therefore it 100% can exceed humans.
Comment · 4 months ago
> defeat any nations that don't discard humans
AGI doesn't do away with nuclear MAD, it just messes with economics and makes many people temporarily jobless. Temporarily because in a literal sense RLVR needs verification to train off of, and a lot of jobs cant be easily checked if theyre done. this includes AI safety people, preschool teachers, psychologists, and probably a lot more, including most of their bosses
Comment · 4 months ago
> Ensuring Chinese compliance would be incredible hard to enforce or to check
consider SALT and other treaties from USSR vs USA cold war. they checked
Comment · 4 months ago
> But every exponential is at a certain point a sigmoid.
strongly, no. its just hard to distinguish them. for example, radioactive decay. cmon
Comment · 4 months ago
> Governments exerting this much control will only end in war
elaborate. regulation -> war, how?
Comment · 4 months ago
we should never make ASI, what I am saying is strongly sane. ASI = not good. AGI, ~human brainpower, can be made safe enough.
> Are there examples of where we have collective decided not to pursue knowledge? Successfully?
human GMO, some bioweopns, I'm sure theres a long list of awful stuff no one wants to exist.
Comment · 4 months ago
source needed. I'm not saying ur wrong, but source needed
Comment · 4 months ago
all easily varifyable tasks can now be solved with money. this is worth paying attention to. math proofs are verifyable -> math proofs are easy now. you can think of other such tasks: cybersecurity, AI R&D/RSI, killing people, 3d-printing helpful tools, maxxing-out human health, manipulation, self-driving cars, anything that can be checked
all jobs in the future will be those can not be easily verifiably done. if you need a team of people to decide if you have been productive, and those people cant be automated, you're in luck.
Comment · 4 months ago
people use different models. I use small local ones for instance
Comment · 4 months ago
Can they add hackernews to their study analysis? And do a measure over time?
Comment · 4 months ago
why is this better than eg matplotlib?
Comment · 4 months ago
strawman fallacy, I can debate anti-subscription arguments much better. I don't think subscriptions have to be bad, but this is unreasonable
Comment · 4 months ago
reminds me of https://www.lesswrong.com/posts/qEbqPitYhWHthwFNu/bun-s-migr.... remember anthropic owns bun
Comment · 4 months ago
I like the quote at the end
"we cannot choose to become idiots"
I've got this revolutionary idea: when downvoting someone, reply to them too. discourse > not