Comment by broabprobe

oh very exciting! Thanks for sharing these sjs382! A bummer the models can't be run on both the GPUs and CPU so you could run much larger models. A year ago I was running gpt120b as it easily fit in 128gb ram. But now I'm only running gemma 26b and wishing I had stuck with 64gb ram since 128gb gets throttled. Not regretting buying 128gb 1.5 years ago though!