mackopes

32" when?

Hey community!

SOGS compression keeps coming up here as a go-to method for reducing Gaussian Splatting model sizes. I put together a deep dive into how SOGS actually works under the hood, with some practical insights on how it can be used in production.

The article stays fairly high-level, but I'm happy to dive into specifics in the comments. I learned quite a bit from implementing my own version of SOGS compression.

Show me another consumer hardware that handles inference and/or training better. How many RTX5090s would you need?

For some time I have a feeling that Apple actually IS playing the hardware game in the age of AI. Even though they are not actively innovating on the AI software or shipping products with AI, their hardware (especially the unified memory) is great for running large models locally.

You can't get a consumer-grade GPU with enough VRAM to run a large model, but you can do so with macbooks.

I wonder if doubling down on that and shipping devices that let you run third party AI models locally and privately will be their path.

If only they made their unified memory faster as that seems to be the biggest bottleneck regarding LLMs and their tk/s performance.

It's weird to see only one word at the time. I don't think this reflects my true typing speed as I have a tiny pause to read the word once it shows on the screen. On the other hand, I'm able to read multiple words ahead as long as I can see them and naturally remove the pause.

I'm not convinced that there is one generalised solution to sync engines. To make them truly performant at large scale, engineers need to have deep understanding of the underlying technology, their query performance, database, networking, and build a custom sync engine around their product and their data.

Abstracting all of this complexity away in one general tool/library and pretending that it will always work is snake oil. There are no shortcuts to building truly high quality product at a large scale.

Damn. Here’s the upvote

Did anyone get a faster click speed than 45ms?

> think of your queries as super human friendly SQL > The database? Massive amounts of data boiled down to unique entries with probabilities. This is a simplistic, but accurate way to think of LLMs.

I disagree that this is the accurate way to think about LLMs. LLMs still use a finite number of parameters to encode the training data. The amount of training data is massive in comparison to the number of parameters LLMs use, so they need to be somewhat capable of distilling that information into small pieces of knowledge they can then reuse to piece together the full answer.

But this being said, they are not capable of producing an answer outside of the training set distribution, and inherit all the biases of the training data as that's what they are trying to replicate.

> I guess my point is, when you use LLMs for tasks, you're getting whatever other humans have said. And I've seen some pretty poor code examples out there. Yup, exactly this.