that doesn't end up with "because publishers want more money"
Well no, because publishers (and authors) do want more money.
When I buy a book I am buying the story.
This is where your analysis goes wrong.
But I never understood why once I have the audiobook I could not be allowed the text version for free.
For the same reason you're not entitled to a free (or even discount) paperback if you buy the hardback, or why I can't return the paperback I dropped in the bathtub for new copy. You're not licensing the story, you're buying a specific implementation of that story. Furthermore different companies have bought the rights to different versions of that story in different mediums, and they all want to earn their money back.
Well no, because publishers (and authors) do want more money.
When I buy a book I am buying the story.
This is where your analysis goes wrong.
But I never understood why once I have the audiobook I could not be allowed the text version for free.
For the same reason you're not entitled to a free (or even discount) paperback if you buy the hardback, or why I can't return the paperback I dropped in the bathtub for new copy. You're not licensing the story, you're buying a specific implementation of that story. Furthermore different companies have bought the rights to different versions of that story in different mediums, and they all want to earn their money back.