While playing a video, select the Settings button.
Locate the Ambient Mode setting in the list of preferences.
Toggle it to off to disable Ambient Mode for all videos on YouTube (in that browser).
It's in the same popup used for video quality and playback speed.
On desktop and mobile devices:
It's in the same popup used for video quality and playback speed.