Comment by daddyo

I'd say a small coding challenge/puzzle/fizzbuzz test is a must-have if you get over a 100 applicants.

I'll always look for curiosity, passion, intellectual honesty.

Depending on the job I'd further look at what they did to distinguish themselves (extra-curricular, self-study, Github projects, etc.)