razorguymania

yes

I think this post is output from the chatbot itself

GPUs can't scale single user performance beyond a certain limit. You can throw 100s of GPUs at it but the latency will never be as good.

Its using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.

Its not using GPUs. Its using Groq's own HW - LPUs