Comment by jakswa

Bonsai 2 27B · Radeon RX 7900 XTX

  - 89 tokens/sec generation with speculative decoding
  - 81 tokens/sec at 20k context
  - 474 tokens/sec ingestion at 20k — about 42 seconds
  - 10.1 GiB peak VRAM with a 24k context window

ROCm 7.2.3 · PQ2_0 · Qwen Q4 MTP, draft length 2

---- versus ----

Qwen3.8-27B IQ3_S · Radeon RX 7900 XTX

  - 79 tokens/sec generation on a short coding prompt
  - 61 tokens/sec at 60k context
  - 53 tokens/sec at 95k context
  - 558 tokens/sec ingestion at 60k — about 108 seconds
  - 19.9 GiB peak VRAM during coding tests with a 100k context window

Vulkan · GSQ-RCO IQ3_S · MTP, draft length 2 · vision projector loaded