djhn

I appreciate the counter argument, even as a chat bot skeptic.

However, there is the fact that you had to know how to poke the machine so it plucks the right vocabulary out of the training data.

There is very clearly no theory of mind: no inherent internalised modelling of how a human of a particular age thinks, speaks and what they do or do not know. These are the most obvious cracks in the “LLMs are (or will be) the superintelligence” narrative.

Research seems to on balance point towards RLHF&RLVR merely increasing subjective sampling efficiency within the pretraining data.

To be fair many of them do have at least a small ownership stake in what their ruling class owns, since their pensions are invested in the public markets as well as, to a more limited extent, VC and PE through various funds and stakes held by public companies.

Yes! How did SARS original bring the face-mask-when-ill culture to Asia, but covid-19 failed to do the same worldwide?

Your comment started out well within the Overton window, but in the second paragraph it jumped out.

Can we call that Overton window defenestration?

This must depend on language. I think that at least for Python, JS/TS, Go, C, Fortran there isn’t much difference in setup with notepad+terminal (or sublime+terminal for myself) relative to whatever the kids are using these days. Other than syntax highlighting and LSP, obviously, but I’m not sure those have much to do with learning how the language or stack work. Could you explain what you mean if I’m misunderstanding something?

Ah, watching that was like soap for a brain sullen with today’s video content.

Wait, what? I’m not sure I’ve ever used an online shop that didn’t allow you to buy things without registering, but then again I do most of my purchases in person and offline.

RE side hustles, as in, reverse engineering?

No, sorry, didn’t mean to come across rude. I wasn’t arguing, I trust that for whatever you were working on, that was indeed low latency and take your word for it. I just can’t think of what field or general area of computing you’re talking about and I’m curious to read up on something I’m unfamiliar with.

I also tested this and couldn’t get the pull down method to work: Spotify still shows its annoying post-screenshot menu.

I’m curious now, what areas of computer science (as opposed to copy paste react development) consider 100ms low latency? Were you working with very big writes? Very old storage?

Meta makes money on advertising. They are not paying 30% fees on ad spend.

Meta does lose revenue when Apple and Google introduce privacy features and decrease ad tracking efficacy. They also hate that unlike the web, where their cookies and tracking pixels provided near 100% coverage of your behaviour, on your phone you might be doing something that they can’t log and analyse.

Constant hallucinations. OpenAI:s latest on max settings. If you are to naively feed say, a short corpus of text to turn it into a parallel corpus in a few different languages, the original text gets subtly mangled and no longer matches the original. Say you have several hundred annotated sentences. Without hand-coding some regex to make sure that each sentence in the source column occurs in the original corpus you’re bound to get hallucinated sentences with an error rate that exceeds 1:100. Whatever you use as the output, JSON or XML, you will end up with columns that just repeat the original instead of translating it, especially for languages that are very close to each other or represent the same language.

Yes, LLMs can be SOTA for NLP, but you’re going to have to use them to write software or workflows that are more deterministic.

Definitions matter. The meanings of words matter.

What you call a model acquiring agency I call plain old software with productivity workflows designed by humans, with deliberate goals. We must separate “model” and an execution environment using a model. [Model] ≠ [A glorified shell script doing API calls in a control flow based on heuristics]. Agents are not AI, they are plain old software. The weights are the model, and that very much remains a static artifact (and pre-post training models haven’t improved much over the last few years).

What you call self improvement is a duck tape hack to imitate persistence and save on inference. Every time you do an API call, anything that needs to be processed is sent to the model. Narrowing that context down saves money. Finding clever ways to do that improves apparent performance and value. The cleverness is still human.

These are all useful innovations on top of LLMs, which remain models that generate text and symbols based on static weights, which in turn represent training data and the provider’s preferences.