crasm
Joined 10 years ago
Karma
174
Submissions
41
Member since
Mar 17
Recent submissions
Comment · 3 years ago
Comment · 3 years ago
LLM inference is GPU bound and VRAM bound. Given quantization however, 2x3090 or 4090 (48GB) is enough VRAM to load 65B quantized llama derivatives (30-40GB). With exllama you can get maybe 20-30 tok/s with dual 4090s on ~1200W. With an M2 Ultra maybe 10 tok/s on ~178W.
(I don't have these devices but have been researching LLM inference performance in the interest of buying a machine for them.)
Advantage goes to M2 Ultra if you ever might need more than 48GB VRAM. I think it's unlikely nvidia is going to release a significantly higher VRAM consumer card anytime soon, since that's what their A100s are for.
Comment · 9 years ago
I'm guessing most people either don't bother with it (I haven't...) or write their own ad-hoc file integrity managers using checksum files and par2. I was about to try something, but decided to go look for existing work instead.
There's a proliferation of backup software, but this is the only file integrity solution I've come across. Surprising, since file integrity is the only way to know if what's being backed up is any good.[0]
Comment · 9 years ago
SEEKING WORK – US/Atlanta – Remote Preferred
I'm a senior CS student at Georgia Tech starting out with freelancing. Preferred technologies are go, java, dart, flutter, and linux shell scripting, but I pick up languages and platforms pretty quickly and have also done some programming in c/javascript/julia/perl6/python.
I'm willing to do pro-bono work for the right client if I can use it as a portfolio piece.
● https://www.linkedin.com/in/demsar/
● [email protected]
Comment · 9 years ago
One thing that makes Anker stand out to me is that they pay attention to the unboxing experience, which I haven't seen from any other company in a non-premium market like cables and car docks.
Comment · 9 years ago
Hey, vmm supports linux guests now.
Comment · 9 years ago
I have an issue with the whole thing. I think they've already lost the trademark to the language.
Correct: The image was enhanced with Adobe® Photoshop® Elements software.
Incorrect: The image was photoshopped.Comment · 9 years ago
So, heptagon of control?
Comment · 9 years ago
I haven't bought or started this yet, but this book on interpreters is on my todo list: https://interpreterbook.com/
You can't match that speed with nvme. Samsung 990 Pro only gets 7.3 GB/s sequential read.
edit: I stand corrected.