Comment by blowski

Just be honest and say "please don't think I'm being rude, but I really need to get on with this work." You can be assertive without lying or being nasty.