I don't know why this is always made a topic, but I never felt offended by "gaijin" or the Chinese counterpart "laowai". No big deal.
However I also don't think that "they don't think it's offensive" has any relevance. Some people think the N-word is not offensive. It still is. Some men don't think calling a female colleague "babe" is offensive. It still is.
However I also don't think that "they don't think it's offensive" has any relevance. Some people think the N-word is not offensive. It still is. Some men don't think calling a female colleague "babe" is offensive. It still is.