I suspect it’s partially an average / regression to mean.
If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.
When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.
The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.
If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.
When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.
The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.