I lock my computer when not near it. If my computer is breached, having user level access of the one account permitted sudo is pretty much Crown Jewels. If you really wanted to privesc you could sniff X11 keystrokes or back door bashrc, but either way even user level access screws me so whatever do what you want after that.
As a result, I just enable passwordless sudo.