By moat I just mean something a solo dev can build/pursue that doesn't get absorbed by vLLM/SGLang in few months. I'm fairly new to this space so not trying to get too deep into kernels or hardware.
Something around inference tooling: benchmarking or eval/regression checks for quantized models or something else. Something that's useful as a standalone tool/project or to learn the space properly. Just curious where there's still some opportunities.
Something around inference tooling: benchmarking or eval/regression checks for quantized models or something else. Something that's useful as a standalone tool/project or to learn the space properly. Just curious where there's still some opportunities.