I've been using OBS for a couple months for live streams on YouTube (see
https://www.jeffgeerling.com/blog/2020/ansible-101-jeff-geer...), and I have had rock-solid reliability, using an external mic interface, an external camera (displayed PIP), and sharing one of two displays during some instructional sessions.
One thing to keep in mind, though—unless you have a dedicated video card, and it's supported by OBS (the list of supported cards on macOS is very thin), your CPU has to do all the compositing and compression, meaning you need a lot of CPU to be able to manage the streaming.
On my 2016 MacBook Pro 13", it barely has the horsepower to do a stream and also run processes that I'm explaining (e.g. manage some VMs, run some database operations, etc.). I had to turn down the compression method to 'ultrafast', which is lowest quality (but still pretty good with 1080p output), and I also use SwitchResX to set my shared display at 1080p 1x resolution (instead of 4K/2x resolution).
One thing to keep in mind, though—unless you have a dedicated video card, and it's supported by OBS (the list of supported cards on macOS is very thin), your CPU has to do all the compositing and compression, meaning you need a lot of CPU to be able to manage the streaming.
On my 2016 MacBook Pro 13", it barely has the horsepower to do a stream and also run processes that I'm explaining (e.g. manage some VMs, run some database operations, etc.). I had to turn down the compression method to 'ultrafast', which is lowest quality (but still pretty good with 1080p output), and I also use SwitchResX to set my shared display at 1080p 1x resolution (instead of 4K/2x resolution).