Your partner hears it, you may hear it when it has moved a bit and the air is leaking, you hear it when you try to go to sleep and the overpressure safety valve is next to your pillow and making a wheezing sound, when you end your sleep cycle and could go to another one except that the mask moved a little and it's bothering enough to wake you up (and then you have to find sleep again), you worry about keeping it and yourself in just the right position that will allow you to go back to sleep, effectively preventing you from falling asleep because of the stress...
And I have a few reasons more,if you really need.