photochemsyn

Alignment is unsolvable for the very simple reason that LLMs are trained on the net content of human-generated literature, which includes the most Machiavellian deceptive manipulative themes human beings have ever come up with - it all there in the weights.

The solution is to put a dumb machine as an overseer of the smart machine, like in a supermax prison. Not good for profitability, but we all knew this was going in the direction of compilers. How much did you pay for gcc or clang this month?

How would it help the world for Aaron Swartz to download 3.5 million research and primary source PDFs?

Example 1: antibiotic and antiviral production technology for developing countries. Even if we limit ourselves to out-of-patent molecules, the primary literature describing the discovery, production, modification and testing of life-saving antibiotics was, at the time of Aaron Swartz, largely hidden behind strict paywalls that typically were accessed through a university library subscription. Access to all that information allows countries and institutions with very limited resources to set up generic medical production pipelines for life-saving medications they couldn’t afford to import - or to become international market competitors to established interests.

Again, that’s just restricting the argument to out-of-patent products that can be generically manufactured with no legal issues. What about life-saving but patented products that are too expensive for poor countries and regions to buy on the open market? Does that added-value IP charge justify unnecessary deaths in such cases?

And about that IP - the vast majority of paywalled research literature was produced with taxpayer funds delivered via government funding agencies, and there’s a very good legal argument that this alone puts it into the public domain. If private companies want to set up Bell Labs 2.0 with no government funding and patent everything they discover, that’s their own capex risk, but just hovering around and scooping up publicly funded discoveries via exclusive licensing deals with universities and researchers makes no sense - instead, patents should be available to any citizen of the government that put up the funds, and research papers should be available to anyone.

As far as the argument that widespread distribution of the knowledge to produce antibiotic, antiviral, and other complex biomolecular products represents a biological weapons risk? Notably that’s why no LLM provider service will help develop technical protocols for antibiotics production, or insulin, or anything similar - claimed dual-use risk fears. This conveniently slows the rise of legitimate independent competition in the pharmaceutical sector, which helps if the goal is maximizing profit margins via monopoly power. Bioweapons production realistically requires nation-state or large-corporation scale resources, and the solution is not obfuscation of basic knowledge but rather international treaties, select agent rules, laboratory operation and equipment purchase licensing requirements, etc.

That’s just one sector - there are at least a dozen other examples besides biomolecular drugs of ‘how it would help the world’ to make all published research literature available to anyone with a network connection, ask any LLM to enumerate them.

This is such a ridiculously complicated subject, there are people who spend ten years in training just so they can handle the complexities of individual instances of this kind of health issue. Why would a corporate media outlet try to weigh in? A crossover between the pharma marketing and the news division seems most likely. Someone said I used AI in my posts recently, and no, I write all my words by hand. But I do use things like this:

Role: You are an expert in the clinical detection and medical treatment of cervical cancer in human females over the course of the 20th century.

Objective: Review the totality of evidence for the cause of these cancers, including but not limited to environmental exposure, viral infection, or any other factor whatsoever that has been reputably liked to the incidence of cervical cancer in human females over the 20th century timeline as reflected in the reputable medical literature.

Ancient Greek philosophy on mind-body developmental balance can help. A physically fit human with no intellectual development and a propensity to follow orders might be the fascist reinterpretation of the classical Spartan ideal, but this would have been viewed as unbalanced aberration. Similarly, producing geeky nerds who can rearrange complicated equations in their head with ease but who can’t run a mile or lift heavy objects is just as undesirable.

This is a historically valid concept, not a convenient utilitarian fiction for the indoctrination of the youth into proper behavior. The idea was that γυμναστική (gymnastikē) and μουσική (mousikē) should be balanced for optimal human outcomes.

Plato’s Republic:

> “Those who devote themselves exclusively to gymnastic become more savage than they ought to be, while those who devote themselves to the other arts become softer than is good for them… The former, if they had no contact with the Muses, become filled with brute force and a mindless boldness; the latter, if they have no training in gymnastic, become cowardly and feeble in soul.”

Notice I was sure to include ‘revival of corporate R & D divisions’ in the argument? Bell Labs is the archetype - if corporations can do a better job of applied R & D, let them. If corporations are upset about publicly funded research generating open-source discoveries that upset the monopolistic apple cart, that’s not an argument about improving efficiency - that’s an argument about maximizing profits and investor return, not for efficient new drug discovery or old drug repurposing.

The real question is, what is the metric of ‘success’ - improved public health for the entire population at low cost, or bloated returns for pharmaceutical investors?

If every drug created with taxpayer dollars at government-funded research institutions was open-sourced, there would be a collapse in monopoly control of drug manufacturing and that would lower prices significantly.

The question is then, if corporations can no longer acquire IP rights to drugs created by taxpayer-funded research programs and transferred to their exclusive control (eg if Bayh-Dole is repealed in the USA), who will invest in clinical trial costs that need to be recouped via a period of inflated pricing?

The answer is government-funded, transparent, and statistically-robust clinical trials of drugs. Once a clinical trial is complete, private manufacturers can compete to produce the drugs at the lowest price by optimizing their manufacturing pipelines against a final product standard regulated by the FDA. If they want to run their own R & D divisions for drug development outside the taxpayer-financed university system to generate exclusive private patents, they certainly can - on their own dime. That’s an investment decision.

If you need to review why this government-linked, tightly regulated system is needed for drugs with clinical effects, just look up ‘patent medicine disasters of the early 20th century’.

We can’t tell if reported anecdotal behaviors of given LLMs are due to (1) one’s engagement history with that particular LLM provider or (2) ongoing variations in the secret system prompt all commercial LLM providers insert or (3) some other variable feature like RAG.

Classify under non-reproducible artifacts of LLM generation.

This is an explainable marketing strategy if the target audience is made up of the military procurement sectors of governments, and is not directed at the mass market tech consumer. The end-user product - monthly subscription to an LLM provider - might not be that profitable. Multi-billion-dollar government contracts for war and mass surveillance might be what their real business strategy revolves around. Getting open-source models banned, that’s another angle.

“Fable was the strongest model on the market” - explain why anyone should believe that claim.

I’ve been trying to track LLM code generation adoption in the critical infrastructure world - as far as I can tell, it’s nill. Zero. Nada. Nobody is relying on these models to write secure code for anything where failure is catastrophic. Planes falling out of the sky. Nuclear reactors going into meltdown. Electrical grids loosing synchronicity. Lots of these BS claims from the marketing and investment crowd, but - it’s just a useful tool for non-critical areas. That’s all it is.

I’ve never used Claude. Why not? Because Claude’s free tier was even worse than Grok! DeepSeek’s free tier is much better. Also, the fact that Claude was hyped on HN like Rust and Go made me suspicious. Why the hard sell and the non-stop promotional effort? I’m mostly interested in scientific programming, the Python and C and C++ seems to work fine, and oh look, Julia!

And LLMs? I’m just going to run open source models on local hardware, it all seems like the 1980s with compilers. Why not just submit by prompts to a high quality model running on so-so-hardware overnight, like the devops cycle with compiling a big codebase? And oh look, nobody pays for compilers anymore, who compiles their code in the cloud?

The funniest part of all of this is that the very people hyping all this - they’re the ones that AI could most easily replace. They have zero specific detailed knowledge - they just orchestrate. Agents are great at orchestration, right? But then, who needs the shareholders, anyway.

If you’re going to get a CS degree, do it in a master’s degree program. Get your undergraduate degree in anything else that involves at least some mathematics, I’d recommend physics, chemistry, molecular biology, planetary sciences - probability, calculus, linear algebra. Engineering is somewhat more on the vocational side, but that works too.

Why? You don’t narrow your scope at the beginning!