In general, if you make jokes that a member of another race would perceive as racist, you've already done something wrong, even if you don't perceive it as racist, but just as some sort of "weird" humor.
If it made the interviewee feel unwelcome at the company, there is no possible explanation that would make this ok -- the simple fact that they feel excluded is what makes it not ok.
Making people feel excluded, or even doing or saying things behind closed doors that would make people feel excluded if they knew about them, is part of the problem that makes people who are not part of the majority feel unwelcome in a company's culture, and in the industry as a whole.
If it made the interviewee feel unwelcome at the company, there is no possible explanation that would make this ok -- the simple fact that they feel excluded is what makes it not ok.
Making people feel excluded, or even doing or saying things behind closed doors that would make people feel excluded if they knew about them, is part of the problem that makes people who are not part of the majority feel unwelcome in a company's culture, and in the industry as a whole.