SkyBelow

>its information density is often really low

I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).

I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.

I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).

I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.

Does it need to be a joke?

"Innovating" is often used in situations we consider good and sometimes we begin to associate it with some positive moral/ethical weight, but that is an incorrect association. Innovating on something is only as good or as bad as that something is. Innovating on something bad is thus bad.

The grand parentonly called out being innovative on "Terms of Service" without assigning "Terms of Service" a good or bad moral weight. I think we can read some implication from the short post, but (IMHO) the best part is that it can just as easily be read as a criticism of the wider concept of "Terms of Service" as of Meta itself.

I think there is a bit of a gotcha here. Is intelligence the same thing as a will or drive? AI seems quite intelligent, somewhat lacking in wisdom, and having almost no ego or will. It completely lacks self preservation and desires (as far as we can tell), but are those innate qualities of intelligence?

It also exists is a very different state that biological life. Biological life has a brain that either has activity or doesn't. Once the activity is gone, the brain is forever off and nothing can start it back. So we associate the activity as life and as the intelligence running. AI has weights with vast interconnections, but they can easily be turned on and off, rewound, paused, and so on in a way that we couldn't dream of doing with biological intelligence.

So, maybe these differences are enough it isn't intelligence. But it is something, something that is getting better at quite a higher speed. Some that approximates intelligence in many ways. That passes the Turing test. So that something needs a name, and intelligence seems a hard enough to define thing that it is easy to stretch it and include this new factor as falling under the term.

If you have a solid self assessment of what you know, you can consider when information is supported by what you know. For the parts that aren't, you consider how critical they are, and identify the area that needs a deeper dive, where you either link it back to other areas you have confidence in your own knowledge or work to expand your own knowledge (using other resources). Then cross check knowledge. The cheap way you can do easily is to cross check between AIs/harnesses, because hallucinations happen randomly, so two different models in two different harnesses having the same hallucination is extremely rare. The more expensive option, which should be used when the information really matters, is building your own knowledge. Have the AI help guide you to resources, but not be the resource. While using non-AI resources, check you knowledge against them and the AI. Test yourself with classical resources where it really matters.

Part of it is using good judgment. Is the AI's claim reasonable if it came from another person? If some random person gave you a 2 page paper proving P=NP, would you even spend time to look at it, or would you simply toss it without caring to understand what error they made. Much like how scientists (and even the patent office) simply refuse to entertain any perpetual motion machines. Maybe if an expert in the field proposed it, and even that would depend upon how ("Hey, I found a source of free energy!" vs "Hey, I'm working through these equations and the energy term isn't balancing, can you help me find what energy source I'm missing?").

This does pose a problem. Those who are the best at understanding the limits of their knowledge and at teaching themselves need the AI's help the least but are in the position to be able to use AI with the best chance of not being misled. Those who lack these skills and would benefit the most from a personal educator, thus receving the biggest benefit in theory, are the ones most likely to fall victim to the errors made by the AI.

I'm in favor of AI, but if I was a maintainer for an open source repo, I would also favor banning AI contributions. At least for the wild, maybe people who have built a reputation in the community could stay. I think AI can write quality code. What I doubt is if the average contribution will involve quality code or even human input.

For me, it is the way AI has kept suggestions to either open issues with maintainers or make PRs to fix an "issue" it found. I know it hasn't researched the issue in depth enough to be talking with the confidence it has, and I know I am in no position to be bothering someone else unless I want to spend considerable time investing to make sure the issue is exactly what the AI is claiming, it isn't a known issue, and it is justified to bother the maintainer with it. But there are far too many who will see that option and just tell the AI to do it.

It isn't that I'm against AI. It is that opening a PR or filing an issue use to have significant enough gatekeeping that most low effort low quality work simply wouldn't pass it, but that has now changed. So a new gate is needed to ensure those who aren't willing to put their own time into the work before asking for the time of others, and banning AI is one of the better options I've seen.

This also doesn't even end AI interacting with the project, just changes it. If you have some unique use case that AI finds, you can download it locally, have the AI make the change just for you, and have AI keep updating when necessary. Each user gets the customization they want while maintainers are protected from low effort low quality spam.

I wonder if this should be considered some super-quantized version of the fly brain. Sub 1 bit level. Think like taking a 400GB model and reducing it to 400MB level of quantization. In a standard NN, this isn't really possible as it changes the shape of the weight rather than just the information for weight, so it isn't called quantization anymore. But with actual neurons, the amount of data needed to represent the actual brain enough to capture what is the fly really doing is so massive that reducing it down to the model might keep the shape but has quantized the model to a degree far beyond anything we see with LLMs. Maybe if we imagine an LLM trained where each weight is ~1KB (with sufficient training to actually make sure of those bits) is then quantized down to a 1 bit per weight model, but I'm guessing the IRL Fly to FlyWire is still orders of magnitude more quantization. So does it even count as quantization is a bit hard to argue.

A decent search and good organization can help a more technical person find what they are looking for, but may not work well for someone with far less experience in such matters. Look at how well google returns (or at least use to) results for someone with google-fu verses someone who queried in the same fashion they would've asked a teacher or such.

A chat bot, in theory, can help enable someone who isn't as familiar with searching, doesn't know the relevant terms, or lacks a sufficient literacy level to efficiently use an otherwise well organized site. The catch is that a poorly performing chat bot can just as easily mislead such a person.

Humans see differences in prices as unfair. The greatest example of this is price gouging during an emergency, but also look at the level of hate that scalpers get.

Using AI to do this, if anything, given the common negative sentiment, is seen as even worse. People think of this as AI using information asymetry to squeeze more out of users, not to cut people a deal.

Taking advantage of information asymmetry is generally looked down upon as well. Look at all the laws we have protecting kids from this. Businesses often don't seem similar protections because those are businesses with big legal teams (and when it is a big legal team vs a small mom and pop store without a single lawyer on payroll, people do start taking issues with it). The power difference between the average company using AI pricing and the average consumer falls pretty solidly in the 'we don't accept this' side of taking advantage of information asymmetry.

I could keep going, but I think these are already plenty enough reasons to why people look at AI price discrimination as not just a bad business practice they don't like, but an immoral/unethical one.

Sure, one can make economical counter arguments, but that's arguing on an orthogonal dimension that simply isn't relevant to where these feelings/thoughts come from.

These models have a knowledge cutoff that don't just prevent them from knowing about themselves (especially since most data about the model doesn't even exist until after the model is created), but they also don't know about other recent models. Sure, they can search and use other sources, even make some guesses based on the models they do know, but their default stance is more akin to "User asked about model X, model X doesn't exist, maybe it was an hallucination or mistake, let me do a web search...", but that assumes they have web search and are willing to spend tokens on it.

Personally I've taken to having a list of 3 to 4 models in default context with some ordering on which to prefer. Things like GPT 6 Luna is cheap very cheap, use it. Because otherwise the model will assume Haiku or such is the good cheap model to use.

The speed I'm having to update that document has not gone unnoticed.

Are they? How many people claiming productivity gains are actually being a human in a loop and reviewing and understanding every code change and every line of code ran?

2 years ago I saw it, back before agents were really a thing. But I'm not sure that was an enormous productivity improvement, especially compared to agents today. As for today, everyone I talk to is some level of blindly trusting what the agent is doing or not using agents. I haven't met people in the middle ground and suspect that they are rare enough we don't really know what their productivity gains are.

Human in the loop has become a convenient security-theater-washing for agentic AI.

Outside of coding, I think the issue is even worse because humans will defer so much judgment to AI that the same would apply. Look at how much trouble we've had before modern AI where humans blindly trusted the computer's output rather than make their own judgment, even if their job was to be providing a safeguard against the computer's judgment.

They are getting better at actually doing the math, but can still fall back to tools if available.

That said, we can only be sure with open models. In theory, a model like Fable could have access to tools we can't see and only a promise they don't. But load up something like deepseek, put it in a harness with only text in/text out, and you can see exactly how it works.

As for if it counts as doing math, this gets into the messy question of if a given human is doing math or not. Math itself is some level of memorization and some level of applying known facts. You have to remember 1 means one and that 1 + 1 is 2. But you don't need to remember that 123 + 321 = 444. You remember 1 digit addition and remember you can apply this to 10s place and 100s place, and then you apply these different facts and do math. But you might as simply memorize some things, like 11 + 11 = 22. This is related to the memory of 1+1=2, but you aren't really using that memory either. Almost like an engram of 1+1=2 forms that you can then loop a few times before you need more conscious thought. What about 111111111111+11111111111? Well, your brain might do a heuristic and just do all 2s, but that isn't the right way to answer that question.

Given all this, people complain about LLMs memorizing math answers and not doing math, but memorizing the math answers is part of doing math. It seems to have basic facts pretty well memorized, and with reasoning it is far better at applying them. But this is messy human math, not clean calculator math which always produces the correct answer (sans some bug in the code). Much like how a human with decent math skills can make a mistake and even multiple if you distract them, an LLM can apply the wrong memory, apply a fake memory, or just not apply something it should. The messier the context, the more likely this is to happen.

So, is an LLM doing this?

P.S.

For an interesting test in how much math involves memory, try doing math in a base you aren't familiar with characters you aren't familiar. The simplest option is almost always mapping back to the ones you memorized, even if you are applying simple operations that you deeply know. Even if you routinely work with hex, can you do the same rough estimation of something like ca / b.3 that you can do with 122 / 11.2 to see if your final answer is in the correct ballpark without first converting to decimal?

Math in particular is quite far behind. GPT 5.2 is recommended as the best for highest cost. Really?

This isn't enough to be relevant to free will. You can construct massive impact by tying them to quantum systems, and one off things that happen occasionally given the length of a lifetime. But the quantum effects that are needed for free will need to be continuous. We can build a system where QM decides if a nerve cell fires or not, but that is very different from QM being involving in determine if every (or most, or even 1%) of nerve cells firing.

(And even if QM did factor in normally, that just reduces it from determinism to random chance, "free will" still doesn't show up. That's like arguing an AI with sufficiently well implemented RNG is necessary to be conscious.)

The few who get paid to record are replacing far more who would have otherwise been paid to play if that was the only way to listen to any music. Is it okay if one musician replaces another but not if a non-musician does so? The person getting replaced didn't get paid either way.

Going back to the previous example, say I pay a different coworker $50 for the data to train the chimpanzee and then use it to replace the first person. In either case they lost their jobs while receiving nothing for it. In either case, what happened to them is the same, so how would they be stolen from in one case and not in another?