> Ironic that these large LLMs are eroding Nividia's moat.
That's not what the quoted part meant. During v3 development they only had access to hardware limited variants of H series GPUs. Those had less interconnect bandwidth IIRC. So, at the time, the low-level wizards that ds employed bypassed the official APIs (i.e. the nvda ecosystem) and hand wrote alternatives to say nccl, to better use those limited GPUs. I remember them publishing some of it as well. It had to do with allocating memory, moving stuff around, etc. Basically bypassing some limitations by going lower than the official APIs support.
There is no eroding of their moat, as long as they sell GPUs. ANd they're selling GPUs like crazy. The moat speaks for itself, if I may :)
That's not what the quoted part meant. During v3 development they only had access to hardware limited variants of H series GPUs. Those had less interconnect bandwidth IIRC. So, at the time, the low-level wizards that ds employed bypassed the official APIs (i.e. the nvda ecosystem) and hand wrote alternatives to say nccl, to better use those limited GPUs. I remember them publishing some of it as well. It had to do with allocating memory, moving stuff around, etc. Basically bypassing some limitations by going lower than the official APIs support.
There is no eroding of their moat, as long as they sell GPUs. ANd they're selling GPUs like crazy. The moat speaks for itself, if I may :)