I am going to save you some time: please do not use herd for this right now.
Firecracker (and by extension, herd) is currently optimized for blisteringly fast CPU and network isolation—specifically. We do not currently support PCIe GPU passthrough at the moment.
If you need fast serverless GPU deployments, look into platforms like Modal. Once we tackle GPU scheduling in herd, I will make a lot of noise about it, but right now it is not the right tool for your specific job.
Firecracker (and by extension, herd) is currently optimized for blisteringly fast CPU and network isolation—specifically. We do not currently support PCIe GPU passthrough at the moment.
If you need fast serverless GPU deployments, look into platforms like Modal. Once we tackle GPU scheduling in herd, I will make a lot of noise about it, but right now it is not the right tool for your specific job.