Growing up to Scottish and Irish Catholics in England, I remember talking to a bunch of Ulster protestants as a teenager when the topic turned to religion - I said "Well, I was raised Catholic but I'm not really a believer of any kind" and the response was "Ah, so you're one of THOSE Catholics!".
Turns out there are a lot of 'those' Catholics.