Matumio

You seem to be describing Grokipedia. In case you never heard of it, well... you'd look it up on Wikipedia wouldn't you.

I guess the fossil fuel infrastructure is actually more expensive? (Drilling, refining, shipping of the fuel: all not needed for wind and solar.) But this infrastructure has already been built and paid for, and now someone with a lot of money wants a return on their investment over the next 25 years. Which is not going to happen in a world where we prevent the worst climate change.

(Also, investment to decarbonize things like cement are not going to happen if fossils stay cheap.)

Quote Wikipedia:

> "Citogenesis" redirects here; not to be confused with cytogenesis.

I had to read that three times. I thought it was a self-referential joke at first. Turns out the name was a joke from an xkcd. Can't argue with that source in terms of authority.

This is like saying with chemistry, amino acids and human DNA you now have the basis to understand human culture.

The problem is one of complexity, the whole being greater than the sum of its parts. Electrical engineering creates a thinking that is opposite to what you need for understanding biology or human culture.

This is relevant because LLMs have basically swallowed all written human culture. Engineering is about creating parts that interact in a clean, well-defined way, like a MOSFET transistor or a REST API. The goal is that that when you combine the parts, and they all operate within specs, you can still predict what the whole system will do.

So saying LLMs are just a step away from the Pereceptron (or an equivalent filter) is mathematically true, but almost useless. It's like saying that a modern 3D game is basically just transistors, logic, CPUs and memory, nevermind the pattern in that memory or how it came to be. You really can't ignore all the cultural inputs and feedback loops if you want any kind of useful understanding what this thing is.

If you're concerned about that, you can concat your JPEG with a few bytes from /dev/random and you'll get the security of whichever is stronger. In practice none of this will be your weakest link.

Yes, but it is a useful insight that both methods try to solve the same mathematical problem. It's better than thinking of LLMs as magic.

When you say "cross-entropy loss" people without stats background go to Wikipedia, take a glance, and adjust their mental model to "inscrutable magic".

Thinking of the main difference as the trade-off in how much CPU, memory and storage is allowed is not really wrong.

The part that is wrong is to think of gzip as a method that might reach similar complexity or generalization. And more importantly, to ignore the advanced way how training data gets curated or generated for (instructed, chain-of-thought) LLMs. But even then. The mental model that the LLM's goal is text compression is not wrong. The question to ask next is what kind of text it is expecting to compress.

You buy your groceries while that piece of your brain has no excuse to make loud demands ;-)

Like if CERN published the discovery of the Higgs boson with 99.9997% certainty, but refusing to tell you how they calculated that number, or what equipment they used and how they calibrated it, in order to prevent other labs from copying their methods.

Or like a machine learning lab claiming SOTA on a benchmark, beating a well-known method that they re-implemented, possibly with bugs, on their private dataset, for millions of compute. But you don't get the source to check, and they don't release any intermediate results or ablation experiments. Aka, from the outside you can't distinguish it from corporate marketing.

Yes but it's annoying to collaborate with a Claude proxy person who thinks to have it all sorted out and who will question everything you say but not really listen, just itching to get back to Claude to confirm or deny what you said.

Compare that to being asked for advice, to explain directly something you know or have done, with the expectation of the person asking to learn something. Like asking why some decision was made in the past, based on what information and context, instead of confidently saying that it was done wrong. Both will eventually get you the same information.

My favourite is a logout button with a logout API that fails. (Not a huge pratical concern, I admit, because it's a local attack.) Nobody ever notices because it still shows the logout screen, which hides the API error toast (if errors were even displayed). The still valid refresh token stays in sessionStorage (or even localStorage) while the app displays "logged out". (Bonus points if you cleared the access token in the error handler but not the refresh token, and on page reload you ask the user to log in again despite having a valid token.)

Or a login form that gets hidden after login, but clears the username and password only when you click "login back in". (Bonus points if the backend also enforces a 5min session timeout "for security".)

If your clients are compromised then what's even the point of backend security. Users will login and do legitimate actions while their compromised client does whatever behind their back, while still looking normal. And the backend can't tell the difference.