Sites have been replacing the actual API request with a HTML5 overlay, precisely because of this - only when clicking "notify me" does the actual permission get requested, with the requisite dialog.
(The rationale is "once the permission is denied, there's no way to bring it up again; a dismissed HTML5 box can be brought up again an infinite number of times")
As for me, I went for WebAPI Manager, which blocks most of the annoyances (Vibrate API? Never saw anyone outside malvertising use that), while still allowing me to manually whitelist sites.
(The rationale is "once the permission is denied, there's no way to bring it up again; a dismissed HTML5 box can be brought up again an infinite number of times")
As for me, I went for WebAPI Manager, which blocks most of the annoyances (Vibrate API? Never saw anyone outside malvertising use that), while still allowing me to manually whitelist sites.