roadside_picnic

It's also harder to explain to your customers why you're the only product they use that has a security problem when an exploit does eventually happen.

You see this already with uptime. If Github or AWS has a major outage, it's in the news and even people that don't understand software realize it's just "something happening today, everything is down".

And, unfortunately, in practice customers don't care about this inverse, when "everyone else is down but not that one product".

> If everything is more bespoke the attack surface is smaller

This is exactly the problem that LLM assisted attacks solve. For a human attacker, the extra work to figure out a bespoke solution is too time consuming to be worth the effort. LLMs can search for vulnerabilities and spit out bespoke attacks like it's nothing. On top of this the bespoke solutions weakness leans heavily into the AI attackers advantage: response time. Even if they can just get your company's single Anthropic key, if they do it at 2am while your one infra person is on vacation, they have plenty of time to extract enough value for their effort.

I agree with the move to hyperscalers. The future (well, actually present) is going to be constantly seeing new attacks happening. The only real solution to this going to be having teams of people and agents constantly watching for these and able to deploy fixes and respond to risks immediately.

For me it's been liberating because I no longer have to worry about what languages I choose to spend my free time on at the risk of missing out on something more practical. For example, I love Common Lisp, but taking time relearn it always felt to come at the expense of something else that might also confer practical benefit. Now, in a sense, all programming languages are equally useless.

I recently did a major move and had to restructure my bookshelf pretty dramatically. The only programming books I kept out of storage where the Common Lisp and Prolog ones. Spending time learning a language that doesn't directly impact how you think about computation itself seems to be more of misuse of time (of course really, when programming truly for fun it shouldn't matter one way or the other).

> for very fat salaries

For a fairly sizable portion of the population (especially the tech worker population), salary/TC is seen as the measure of someone's value as a human being. I've had acquaintances who really, sincerely struggled to understand the concept than an engineer making 1/N what an engineer at Meta/Google/etc does could be overall a better engineer. The typical reply has been something along the lines of "but if they're better, why don't they just make more money?"

They really couldn't understand when I tried to explain that, in my experience, nearly all the really brilliant and interesting technical people I knew tended to be making less, because they would rather work on interesting problems with a high degree of control than play leveling games at a large corporation.

> I have created.

Are you working on a team? I've found that for solo projects this works but, even when you own the design of your piece, integrating with the work of several people doing this ultimately leads to even the part you initially designed no longer working exactly as you remember it.

A good chunk of my time now is spend working with an LLM to understand how other parts of the code base work and how they relate to and interact with the product features I manage. But I'm still on a small enough team that this is possible. I'm not surprised at all that an engineer at an org the size of Meta would not even be able to perform this level of work in understanding what's happening.

> Layer on top that some offices are owned by private equity!

I used to be very regular with dental visits, and have had an enormous amount of work done over the years. Then I went to my first PE owned dentist (by accident) and it was such a horrific experience I've basically avoided dentists since.

There used to be some reasonable balance between the dentist wanting to make money and wanting to build a secure custommer base/happy clients. PE dentists are just money extraction machines with little to no concern for how much pain is caused getting money out of your wallet.

I've never understood why "Hacker" News so frequently gets "But why though?" comments at the top.

The entire history of innovation is filled with people doing something just to see they can get it to work, even if badly, and then people continue to iterate on that until it works better, then works well, and then is so obvious people would never even question it. But it all starts with someone doing it to scratch an itch.

Neural networks, the foundation of our current AI revolution, used to fit well into the "neat, but practically useless" category.

Sure there are countless "but why though?" experiments that don't pan out, but that's just the cost of exploration. There can be no step-function innovation in a world where people only do things that make immediate practical sense.

> and I would not call it cheap

It is compared to the cost of making an authentic ramen broth at home at the scale of feeding 2-4 people. Pho has a similar property. I've made homemade pho broth once and it was very hard work, fairly costly, and it resulted in something that was pretty good, but not comparable in quality or price to something I could have bought in a restaurant down the street. (admittedly typically also cheaper that ramen)

> We do this and then no one reads it.

This to me is an anti-pattern that I see everywhere because we're still in the early stages of understanding how to adopt these things and create real agentic workflows.

Why should humans read all that documentation if they're ultimately not going to be the ones implementing it?

I see similar patterns like this everywhere, people asking questions about a PR that an engineer then copy pastes to an agents and the copy pastes back to the PR. It's obvious that humans are still in the loop in awkward places.

I've pointed this out before, but web development went through similar growing pains where we did all sorts of weird stuff because we hadn't quite figured out the patterns that work.

Right now nobody really knows what works and the tools we're using are still changing very rapidly. 6 months ago almost nobody was using agent harnesses and thinking about agents writing all of your code was preposterous. Today even the hardcore, AI-skeptic people on my team still have agents writing the vast majority of their code.

It's very clear to me that we're still stuffing agentic behavior into workflows designed for humans. It shouldn't be at all surprising that these patterns feel weird and sometimes broken. It has nothing to do with "winning over the average Joe" but rather these patterns evolving so they actually make sense.

> from 40 to 50 tok/s generation

That's actually much better than I would have thought!

Thanks for the answer and it does make this approach make more sense as a budget solution to running larger models locally.

What's the tokens/sec you're getting on that setup (genuinely curious because it's a setup I haven't actually run myself)?

> no other widely accessible path for young people to build a future

This is exactly right. Widespread gambling has a fundamental nihilism baked into it. If you believe hard work and frugality are the best path to success, then gambling doesn't feel like a good use of your income and energy.

But if you feel like your dreams are falling out of reach despite your hard work, if it starts to look like even big investors are really just playing a game (often with loaded dice) rather than acting on fundamentals, if grift seems like a more reliable path to success then hard work and ingenuity, then all of a sudden gambling seems a lot more sensible. Even if the odds aren't in your favor, you can at least get lucky when you gamble. When the game feels rigged its easy to believe that hardwork is destined for failure, whereas even loaded dice have some probability of landing in your favor.

If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use.

I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s

While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX.

The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have.

That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself.

Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.

> Yet those 2-3M channels get the lions share of the views.

...because they're directly tied to YouTube own revenue.

As someone who creates a fair bit of content, and am fortunate enough not to have to worry about the revenue from that, I deliberately try to minimize monetization because it ultimately becomes the main driving force behind your content creation.

Monetization as the only model for content creation leads to an incredibly safe and boring world while simultaneously meaning every piece of content you interact with is trying to extract money from you the viewer.