Comment by silentrob

I think you are describing TREC[1]. Over on the chatbot arena, there is The Loebner Prize[2], also interesting, but in general the systems are over learned.

[1] http://trec.nist.gov/ [2] www.loebner.net/Prizef/loebner-prize.html