jgord

They could just use their frontier model to trade the stock market, right ?!??!!

its not just math .. its math, science, engineering, medicine, tech .. ie. everything.

We need to ensure we dont outsource thinking itself - yes its great to have a lot of our knowledge encoded in books, and in the weights of LLMs [easier to access ]

Do we even 'have' a technology if it doesn't reside actively in some human minds ?

Some of math is handed down to apprentice mages, and not encoded well in books [ perhaps books dont encode intuition well, Ive had many cases where someone has explained something not in the textbook, a trick they learned from someone else ]

Apprentice mages only become mages by grappling with hard problems, asking questions, going on long detours, discussing/arguing with master mages, trying lots of tough thought experiments - all valuable "training".

Training using the AI might lead to a golden age where more mages learn wider and deeper, and the subject flourishes and more humans understand the tech.

The danger is using the AI makes us lazy - we dont grapple with programming syntax, draw math diagrams slowly by hand, check edge cases, try to prove things ourselves the slow way and build intuition in the process.

Another issue is ownership and access. Already RAM GPUs and soon CPUs are harder to get hold of due to competition with the hyperscalar datacenter buildout. This is an example of economic inequality effectively stealing resources from the mainstream.

Will math remain open or will it be owned by the frontier models ? With Math and science journals Elselvier almost had a stranglehold, bullying universities to pay massive bulk subscriptions - fortunately Arxiv solved some of that. Can I even own a copy of a music track, tv show or movie these days, when there it is so much more profitable for companies to rent it out pay-per-view or monthly ? There is every financial incentive to slurp up the whole of Arxiv, github and every science textbook into a frontier model and rent out access via pay per token. I no longer need to go to stack overflow to look up a simple algo, google can give me a pithy summary, so its likely that the LLMs will steal traffic from github, or github could be drowned in the volume of vibe code. Is there any point buying a textbook or going to university when the AI covers the material ? If no universities then we dont have a physical space for science and tech to be discussed. That activity might go online .. or may disappear altogether - we already have less socializing in person due to the rise of social media.

We have several "alignment" problems : humans arent aligned with each other, frontier models are aligned with profit-making which may differ from the common good, the race is on and so frontier models are more aligned to short term shareholder / VC value than to longterm benefit to humanity as a whole. These mis-alignments may have always been there, but are multiplied by the power and reach of new AI technology.

yes ... I do think that if humans stop thinking hard about existing problem domains .. they will lose that ability.

From which it then follows that _all_ technology and science become magical incantations dictated by a black-box oracle, whom everyone trusts and no one understands.

That could happen in a couple of decades.

I genuinely am interested in what other people see as AI psychosis.. was not being sarcastic .. I have my own impressions, but want other field-reports.

I guess my counterpoint is bluntly - no-one will not be using the tool at their day-job.

It requires so much extra discipline to think for yourself and not use it, you will be forced to use it in your job or be seen as a Luddite and your ability to think for yourself in the domain will diminish over time - your domain thinking muscle will gradually atrophy from lack of use.

Why would any of us bother thinking hard, when there is no financial incentive to doing so ?

Your analogy fails with shovels and excavators .. because General synthetic intelligence is not a single purpose narrow 'tool'. near-future AI is equivalent to an alien intelligent species competing for resources, or equivalent to a cheaper smarter non-human 'outsource worker'.

Can you describe the symptoms of the AI psychosis youve observed ?

Yeah, considering how hard it is to food-diet or social-media-diet .. I think it requires immense resolve to LLM-diet or use it in a non-binge way that develops ones own depth of understanding.

Its going to require just far too much mental energy to push back against an LLM that is correct 92% of the time.

edit : removed air quotes around AI psychosis.. that might have read as sarcasm, which was not intended

It might be better to view the near-future AI as an emerging intelligent alien species.

Regardless of whether it is 'conscious' or not, Roman Yampolski makes the point that AI may or may not have human-kinds best interests at heart [ hell, even humans aren't aligned ]

Can we control or trust a counter-party that has intelligence at a level near or exceeding the smartest humans in all domains ?

I dont even think we can even have a safe sandbox in which to run an AI .. and frontier model training labs are certainly not air-gapped.

I think 95% of the population wont get a choice of how AI is framed - the big money pushes how it will be framed for them, and most people have no training in a hard discipline like math or science, as a vaccine against that marketing push. t People will be corralled into trusting everything an LLM says as gospel, will be required to use if for almost all jobs, will have no actual choice in the matter - everyone who suggests otherwise is a luddite. [ Terence Tao is being painted as a luddite currently, for raising the concern of whether human-math will even survive ]

Its probable that AI will roll out much faster than humans can develop that skill of using it in a way that increases not decreases human knowledge over time.

The current slew of excellent LLMs will probably cause a decades long dip in human cognitive ability.

If social media on smartphones had an impact of ability of students to concentrate, LLM AI will likely have a much greater negative impact on education.

There are trillions of dollars of investment pushing for humans to trust whatever an LLM says, and very little money behind encouraging people to tackle hard problems themselves and train themselves to think.

I have to say honestly, that AI looks like an apocalypse for human thinking [ Im glad Tao is warning people, I just dont think many people will listen ]

that might be a good thing, on balance.

Pithy excerpt :

> The fix isn't to abandon AI. The fix is to use it differently. To use it in a way that sharpens us instead of softening us. To use it in a way that makes us think harder, not less.

maybe the fix is to _not_ use AI in your domain of expertise, or at all.

or, the fix could be to use narrowly trained AI - as Roman Jampolsky argues. Rather than train an AGI / ASI on the whole of human experience on the internet, perhaps train the models for narrow use - on narrow focused domains like microbiology / nano-materials etc

or the fix could be to make training AGI illegal - a regulated munition.

That might be a way to avoid creating a new intelligent species, which is competing with humans for compute and other resources... while making us dumber via outsourcing all our thinking in the short term.

Raises a good point .. in, oh, such a bad way.

it does feel AI-written, sadly.

Have been using reader view more and more .. mainly due to font readability, I like sans.

Ctrl-Alt-R in firefox

I was amazed at how much better compression webp offered over jpg and png.

Using it in a web-browser 360 panorama viewer enabled pretty substantial latency and storage improvements.

close to 4x iirc.

ditto large "xray" style orthographic images of building scans generated from pointcloud laser scan data.

avif was even better compression for some cases, but seems to take a long time to generate, and less widespread browser support. Will look at jpeg-XL .. but Im guessing it will be a while until it has widespread support