Comment by bigcorp-slave

It depends on your application. If you are running on a smartphone, or on an AR headset, or on a car, or on a camera, etc, you generally do not have the latency budget to wait for multiple frames and run at high batch size.