This is a great project: the code is super neat and the write-up very clear. I like the sousveillance aspect of it!
For a project I needed a low-latency RTSP stream as well. When reading a video stream with OpenCV, the default video buffer is quite big, which, when filling up, makes the video lag behind a second or two. It then becomes impossible to perform any interaction on it.
I wasn't familiar with the setting you use to overcome this: setting cv2.CAP_PROP_BUFFERSIZE to 1 on the VideoCapture.
I am not sure, but you might get even lower latency by turning to OpenCV's GStreamer support. For me the trick was:
gst = f"rtspsrc location={video_url} latency=0 buffer-mode=auto ! decodebin ! videoconvert ! appsink max-buffers=1 drop=true"
self.video = cv2.VideoCapture(gst, cv2.CAP_GSTREAMER)
When testing, I also found out that the codec and image settings of the camera matter. With a h264 stream, the images came in batches of a number of frames, whereas MJPEG provided a more constant image stream with lower latency. Lastly, disabling 3D noise reduction also removed some delay.
For a project I needed a low-latency RTSP stream as well. When reading a video stream with OpenCV, the default video buffer is quite big, which, when filling up, makes the video lag behind a second or two. It then becomes impossible to perform any interaction on it.
I wasn't familiar with the setting you use to overcome this: setting cv2.CAP_PROP_BUFFERSIZE to 1 on the VideoCapture. I am not sure, but you might get even lower latency by turning to OpenCV's GStreamer support. For me the trick was:
When testing, I also found out that the codec and image settings of the camera matter. With a h264 stream, the images came in batches of a number of frames, whereas MJPEG provided a more constant image stream with lower latency. Lastly, disabling 3D noise reduction also removed some delay.