Comment Comment by walterbell 200+ comments, https://news.ycombinator.com/item?id=42897205> This runs the 671B model in Q4 quantization at 3.5-4.25 TPS for $2K on a single socket Epyc server motherboard using 512GB of RAM. parent walterbell 1 year ago
> This runs the 671B model in Q4 quantization at 3.5-4.25 TPS for $2K on a single socket Epyc server motherboard using 512GB of RAM.