It apparently existed pre-Tolkien [not really surprisingly as it's one of those weird English plural forms that can go multiple ways] and there's at least some evidence that he retconned it as a deliberate choice. e.g. https://jakubmarian.com/dwarves-or-dwarfs-which-spelling-is-...
ADDED from the link:
Tolkien himself admitted that “dwarves” was a misspelling. In a letter to Stanley Unwin, the publisher of The Hobbit, he wrote:
"No reviewer (that I have seen), although all have carefully used the correct dwarfs themselves, has commented on the fact (which I only became conscious of through reviews) that I use throughout the ‘incorrect’ plural dwarves. I am afraid it is just a piece of private bad grammar, rather shocking in a philologist; but I shall have to go on with it."
ADDED from the link:
Tolkien himself admitted that “dwarves” was a misspelling. In a letter to Stanley Unwin, the publisher of The Hobbit, he wrote:
"No reviewer (that I have seen), although all have carefully used the correct dwarfs themselves, has commented on the fact (which I only became conscious of through reviews) that I use throughout the ‘incorrect’ plural dwarves. I am afraid it is just a piece of private bad grammar, rather shocking in a philologist; but I shall have to go on with it."