Comment by vegesm

I don't really get the first part of your comment. You say an NN is "just" compression + kNN and does no representational learning. But finding a compression (a transformation in other words) that makes kNN feasible on the data is exactly what people mean when they say it finds a hidden representation. It is a highly non-trivial task: e.g. simple distance in pixel space between images would get you useless results.