> Not to mention no one serious is serving this on 8xB200 instead of multiple nodes: the vast majority of Moonshot's inference work is focused on PD-disaggregation
The GPU price discourse is absurd, but many are serving models on single node setups when the model fits
The GPU price discourse is absurd, but many are serving models on single node setups when the model fits