eggbrain

Everyone says they are packing up. But most of the people I’ve seen complain either are still on the site they complain about, or left briefly just to realize that other sites aren’t any better and came back.

Right now you are right -- even if I ran my local LLM all day, the quality is not nearly as great, and it runs slowly -- so I use the tier one AI subscription services as they are faster and smarter. But that might only be true for a limited amount of time, and a limited number of circumstances.

To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.

Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.

So right now you are right, but who knows how long that will last.

Performance optimizations don't just help the western labs, they also help with running more powerful/useful LLMs locally.

If local LLMs get "good" enough, people will soon paying for subscriptions to ChatGPT and Claude, which hurts their revenue.

I can imagine the attention economy sinking, but I struggle to believe that we as a species won't replace it with something potentially worse.

E.g. before the internet there was TV, and advertisers have filled up that space for decades -- before that magazines and newspapers did the same. At this point, can humans believe that they are not being farmed for their eyes?

That being said, I do think the internet potentially becomes basically plumbing -- something that we don't really touch (as our AI agents or whatever comes next are the ones that absorb the content there).

In the paper (section 5.1), they actually tried to abstract high level directional insights into the prompt in order to see if that helped, and they basically found it underperformed a prompt that didn't have those insights at all, implying that directional guidance perhaps over-constrains things.

Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing.

Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlimited steps of course, because you have a finite amount of compute.

So each agent goes off, and by the end, Agent 1 got to 90% accuracy, Agent 2 got to 80% accuracy, and Agent 3 got to 89% accuracy. Agent 1 wins, of course.

But then you look at the refinement steps, and after 2 steps, Agent 1 was _already at_ 90% accuracy. So the agent spent the next 8 steps basically not moving at all. Agent 3 on the other hand, perhaps was continuously climbing in accuracy at every refinement step, but hit step 10 and had to stop.

Now because you recorded every step from every agent, you know what you'd do differently next time -- you'd not allocate as many steps to Agent 1, and give Agent 3 more steps, because perhaps that might result in Agent 3 coming up with a better answer.

From my understanding, that's what they built in the form of a "search" controller -- a way to evaluate automatically and reapply how you could allocate resources more effectively, when applied to a new problem.

But I guess my misunderstanding is how applicable the search controller is when applied to new problems -- just because one pathway stalled early for one problem, doesn't mean it would work for another?

I've stopped waiting for sites to do what I want in terms of features and functionality -- their incentives and mine very rarely align.

I think the solution here eventually will be a search engine that lives on your personal computer (it's also what I'm building currently). Ingest content from the web -> sort/filter/view it locally as you choose.

Note -- if you do this, watch out for requesting access to "all tlds". They send you two emails per TLD -- one for your pending state, and one for your approved/rejected state. I suddenly had 1k+ emails flooding into my inbox, until I found the setting on their website to disable emails.

This is actually where I see software going in the short term -- cloud moving to local.

A few years ago, if you wanted translation, you'd use Google Translate. If you wanted to search the web, you'd use Google search.

But for a few gigabytes, you can now install nllb-200-distilled-600M, and get translations for almost any language locally. You can have your computer crawl the web, create abstracts and categorizations for websites, and build search exactly as you want it.

The main limiter now is hard drive space (and to an extent, local compute) -- but right now it feels like the 70s again where the terminal into a remote server turned into building applications locally.

You may need to select a smaller model, but an M4 with 16GB ram should work!

DHH was not the point of my comment (in fact, I said if the author had focused on him they'd have more of a point), so this unfortunately feels needlessly attackative.