Comment Comment by tacoman I am using the exact same model. Ryzen 5600G w/32GB and an Nvidia P40 w/24GB VRAM20/33 layers offloaded to GPU, 4K context. Uses 25GB system RAM and all 24GB VRAM. 5-7 tokens per second. parent tacoman 2 years ago
20/33 layers offloaded to GPU, 4K context. Uses 25GB system RAM and all 24GB VRAM. 5-7 tokens per second.