CoolGuySteve

They're not a particularly fast card compared to something like a 5070, they have lots of VRAM.

That's why they were cheap before.

Also "Some stupid game", who woke up and made you king of hobbies.

"give them difficult tasks beyond your own intellect?"

Lol no, I've yet to find a model with those properties. Sounds like a fast track to AI psychosis.

The domain I work in doesn't have enough public documentation for these models to be particularly helpful without a lot of handholding though.

I've owned this GPU for 5 years already, it's fine

I'm using the qwen cli tool with a duckduckgo search skill that I made Claude write. It's like bootstrapping I guess

Once it can search for factual information online the smaller model size becomes less noticeable

I get about 35-40tok/sec on a 3090.

It's actually about the same speed when accounting for how much more responsive my system is to Anthropic's saas infrastructure

I'm finding qwen 27b is comparable to sonnet but my self hosting has about 5 more 9s than whatever Anthropic's vibe coding. I also don't have to worry about the quality of the model I'm being served from day to day.

Probably the most damning fact about LLMs is just how poorly written their parent companies' systems are.

The soundtracks for SimCity 3000, 4, and the 5th one titled just "SimCity" are written specifically to be played while doing some fiddly micromanagement tasks.

Cool now use the reference on another thread.

No because as far as we know 26.04 won't enable zswap or zram whereas Windows and MacOS both have memory compression technology of some sort. So Ubuntu will use significantly more memory for most tasks when facing memory pressure.

Apparently it's still in discussion but it's April now so seems unlikely.

Kind of weird how controversial it is considering DOS had QEMM386 way back in 1987.

I had a memory management problem so I introduced GC/ref counting and now I have a non-deterministic memory management problem.

Sure but what did those guys do all day? 400 people is a lot of people

Even at 100k employees I’m still dumbfounded by that number. What do all these people do all day?

It's really funny that after going through all those fonts it landed on Ubuntu Mono for me which is what I use anyways to code in my terminal.

I wonder if it's Stockholm syndrome or if I really do prefer it. It's a totally fine font, I've never felt the need to change it. All the default open source mono fonts seem completely adequate I suppose.

> The Chinese models are open source because they are not state of the art

I think geohot is burying the lead in this text in his post with a lot of speculation.

It's not not that these specific models will become closed it's that the hardware/hosting vendors have an incentive to train models where inference is custom tuned to their chip's dimensions and VRAM.

The Chinese models do a great job of showing what's capable on consumer/prosumer hardware because of export restrictions but anyone entering the hardware space has the same incentives to undercut the frontier labs so they can sell more hardware.

It's also not clear if being at the forefront of inference quality really matters. The open source models appear to be doing a fine enough job of keeping up even if they're a few months behind. So it seems like there's not much of a technology moat for these labs other than the capital costs of training/serving.