This was kind of the target of the 2016 Blizzard Challenge (http://festvox.org/blizzard/blizzard2016.html), as the training data was children's books (clean audio, 'reading voice' prosody, etc).
Some of the voices that came out of that were incredible.
Some of the voices that came out of that were incredible.