Comment by mbanzi

Depends on the protocol they use on the USB connection. Serial over USB has its quirks. Using a Leonardo, a Due or a SAMD21 based Arduino allows you to implement something like HID which should give you better latency