Comment by fancyfredbot

Can someone knowledgeable please put these numbers in context with a comparison to an H100? For example they say 1053 tokens per second throughput with batch size 4 on llama2 70b. Is that good?