A 17 million parameter model (~Resnet50) takes more than 50s proof time. Is this on top of the inference time?
I can see some niche applications for this system, but I am very skeptical it's ability to handle larger models (100M+) and the ability to and it's scalability when there are increased demand.
I can see some niche applications for this system, but I am very skeptical it's ability to handle larger models (100M+) and the ability to and it's scalability when there are increased demand.