This is actually a common misconception. In blind testing, the average listener reliably prefers the more accurate speaker. There is a talk [1] by Dr. Floyd Toole that is an excellent crash course on the science of sound reproduction and listener preference.
[1] https://www.youtube.com/watch?v=zrpUDuUtxPM