Having attended a top Uni and a not-so-top Uni (long story; two different degrees), the single reason people aim and attend for the top Unis is the attached prestige, the network of contacts you will make (and hit up via alumni), and the doors it then potentially opens. The best tutors by far, who made themselves more available, who seemed more grounded than pure ivory tower academic, were in the less-than-top Uni.
My experience, ymmv.