Comment

Comment by ismailmaj

I don't think it needs higher VRAM, the whole point is horizontally scaling.

Most models have moved to MoE and expert routing which is done over infiniband, so tensor parallelism where Tenstorrent was really weak at isn't used much, I can see it as a viable alternative where latency isn't a concern but cost is.

Also even if they wanted to increase the capacity on their GDDR, they wouldn't be able to since they are limited by the GPU memory channels.

parent ismailmaj 20 days ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton