Zondartul

It is easier to trust in the correctness and reliability of an LLM when you treat it as a glorified NLP function with a very narrow scope and limited responsibilities. That is to say, LLMs rarely mess up specific low level instructions, compared to open-ended, long-horizon tasks.

In my unqualified opinion, LLMs would do better at niche languages or even specific versions of mainstream languages, as well as niche frameworks, if they were better at consultig the documentation for the language or framework, for example, the user could give the LLM a link to the docs or an offline copy, and the LLM would prioritise the docs over the pretrained code. Currently this is not feasible because 1. limited context is shared with the actual code, 2. RAG is one-way injection i to the LLM, the LLM usually wouldn't "ask for a specific docs page" even if they probably should.

At some point our understanding of fundamental reality will be limited not by how much the physicists have uncovered but by how many years of university it would take to explain it. In the end each of us only has one lifetime.

Comments that explain the intent, rather than implementation, are the more useful kind. And when intent doesn't match the actual code, that's a good hint - it might be why the code doesn't work.

Ask stupid questions, receive stupid answers.

What does "speculative" mean in this case? I understand it as CPU-level speculative execution a.k.a. branch mis-prediction, but that shouldn't have any real-world effects (or else we'd have segfaults all the time due to executing code that didn't really happen)

Normal welding needs heat to melt the metals. Cold welding happens without heat. Two metal parts will cold-weld on any smooth, touching faces if the air molecules that keep the two separated disappear.

Cold welding is unintentional, spontaneous joining of two metal parts in vacuum. You don't want that to happen, especially if the parts are meant to move.

Normal welding is intentional application of heat to partially melt two parts at the seam, so that they "mix" in semi-liquid state and become one part when they solidify. Welding may or may not use a third material (solder) to aid the process.

ARC problems are too hard for me. I'm no longer sure I'm generally intelligent.

By "learns" do you mean "just shove the entire codebase into the context window", or does actual training-on-my-data take place?

To be fair, once someone has physical access to the machine, them having full access is just a matter of time and effort. So at that point it's security-through-too-much-effort-to-bother.

Future NeuroLink collab? Grab the experience of qualia right from the brains of those who do the experiencing.

If we figure out AGI, that still doesn't mean a singularity. I'm going to speak as though we're on the brink of AI outthinking every human on earth (we are not) but bear with me, I want to make it clear we're not going jobless any time soon.

For starters, we still need the AI (LLMs for now) to be more efficient, i.e. not require a datacenter to train and deploy. Yes, I know there are tiny models you can run on your home pc, but that's comparing a bycicle to a jet.

Second, for an AGI it meaningfully improve itself, it has to be smarter than not just any one person, but the sum total of all people it took to invent it. Until then no single AI can replace our human tech sphere of activity.

As long as there are limits to how smart an AI can get, there are places where humans can contribute economically. If there is ever to be a singularity, it's going to be a slow one, and large human AI vompanies will be part of the process for many decades still.

You can say "Bullshit". LLMs bullshit all the time. Talk without regard to the truth or falsity of statements. It also doesn't pressupose that the trueness is known, nir deny it, so it should satisfy both camps; unlike hallucination which implies that truth and fiction are separate.

I wonder if there is some sort of transition between recalling declarative facts (some of which have been shown to be decodable from activations) on one hand and completing the sentence with the most fitting word on the other hand. The dream that "hallucination" can be eliminated requires that the two states be separable, yet it is not evident to me that these "facts" are at all accessible without a sentence to complete.

My hunch is that since LLMs are trained on a per word basis (okay, per-token), vacuus verbosity is overrepresented.

If you have one normal sentence and one overly verbose, the latter will have more tokens and therefore more weight.