treesciencebot

we build our sandbox just for this use case, fal.ai/sandbox. take the same image/prompt, and compare across tens of models.

the main question is going to be software stack. NVIDIA is already shipping NVFP4 kernels and perf is looking good. It took a really long time after MI300X's that the FP8 kernels were OK (not even good, compared to almost perfect FP8 support in NVIDIA side of things).

I will doubt that they will be able to reach %60-70 of the FLOPs in majority of the workloads (unless they hand craft and tune a specific GEMM kernel for their benchmark shape). But would be happy to be proven wrong, and go buy a bunch of them

fal | Growth Engineer | San Francisco (on site 5 days/wk)

Help us scale generative‑media infra: hack demos in the AM, pitch partners over coffee.

You’ll build quick client libs & microsites, run data A/Bs, write content that drives sign‑ups, and hand‑hold new devs.

Need: Python, JS/React/Next.js, SQL; speed, ownership, love for gen‑AI. Get: strong salary + equity, platinum health, unlimited “build‑something” stipend and most importantly a seat at a rocketship.

Shoot a link to something you’ve built to [email protected]

GH200 is nowhere near $343,000 number. You can get a single server order around 45k (with inception discount). If you are buying bulk, it goes down to sub-30k ish. This comes with a H100's performance and insane amount of high bandwith memory.

at a relatively new high-rise in rincon hill, AT&T still charges 80-90$ for 1 gig symmetrical (same with webpass/xfinity).

For traditional LLMs this might be true (especially large MoEs at bs=1) but I highly disagree with "multi-modal models" phrase since most of the models that output in other modalities are generally compute bound. Which means less flops will make the experience so much worse (imagine waiting a couple minutes for an image and hours for videos).