om8

> could you guys at least use some of it?

Why would you think that companies want to use this information for your good?

I don't think that they care. Orders of magnitude more resources are poured into A/B retention testing than to user's actual quality of life improvement

That’s a good thing in the long run. It was time for us to grow up and pay our debts.

Existential risk can be another big reason

No, thanks, I have my own

Could've been better if GGUF implemented QTIP format. GGUF representation is a major limitation for llama.cpp quantization performance

> If you’re just using a code book to reconstruct a f16 model the only savings you can get are in sending it over the wire.

That’s why you need to use efficient gemm kernels like FLUTE for inference. They are ~as good as what you can do with ternary quantization.

If you want sub-2 bit llm, get one that’s already trained in higher precision, and compress it with something like YAQA/QTIP with finetuning or PV-tuning + AQLM/HIGGS

Ternary quantization does not make any sense. Vector quantization and trellis based methods are better in this region for PTQ.

Why int4? There are a lot of superior 4 bit formats like nf4 from bitsandbytes.

> So how can I regain my ability to focus?

Getting diagnosed with ADHD and taking meds did the trick for me.

Not sure how relevant this is, since inability to focus was always an issue for me.

> manufacturing companies making the flimsiest, cheapest, plastic crap to save 1/3 of a cent on every mop they produce. Designed to work for the least amount of time before needing replaced

We live in a world with such companies, and we can still buy quality things. If there is a demand for the purely-human generated texts, they will be around. Perhaps a lot of people around you will read ai text instead, and you'll get upset because of it, but it's their choice. You'll still have your thing

Oh, yes, you are right about watts. My bad

> volt-amperes are technically the same unit as watts

volt-amperes are joules

Cool way to think about GWh/year:

  1 GWh/year = (10 ** 9) / 24 / 365.25 / (10 ** 6) MW = 0.11 MW

  70 GWh/year = 8 MW
  1755 GWh/year = 200 MW
  252 GWh/year = 29 MW