Comment Comment by Mistletoe Can anyone comment on the TinyBox they are taking preorders for?The tinybox738 FP16 TFLOPS144 GB GPU RAM5.76 TB/s RAM bandwidth30 GB/s model load bandwidth (big llama loads in around 4 seconds)AMD EPYC CPU1600W (one 120V outlet)Runs 65B FP16 LLaMA out of the box (using tinygrad, subject to software development risks)$15,000 parent Mistletoe 3 years ago
The tinybox
738 FP16 TFLOPS
144 GB GPU RAM
5.76 TB/s RAM bandwidth
30 GB/s model load bandwidth (big llama loads in around 4 seconds)
AMD EPYC CPU
1600W (one 120V outlet)
Runs 65B FP16 LLaMA out of the box (using tinygrad, subject to software development risks)
$15,000