Comment by thehamkercat

update: coreweave/fp8 is at 191 tps, launched this morning, but really bad cache hit rate (~60%), coreweave is good for privacy but let's hope they improve cache