You're probably correct here. The part people find a little unsavory is not that it's somehow eroding the middle class. It's the idea that HFT can act as an unnecessary intermediary, essentially taxing a transaction that otherwise didn't need to be disrupted and at scale the amount extracted becomes material.
Imagine your neighbor owned a Ferrari and you told him one day you were going to buy a gallon of milk at the store. "On sale for $3.99!" you say to him. Now imagine he sped past you on your way to the store and when you arrived there he had bought all the milk at $3.99 and was selling it in the parking lot for $4.01.
You wouldn't necessarily be ruined financially paying $4.01 instead of $3.99. The cost is negligible. But you would probably think he was kind of a jerk.
Imagine your neighbor owned a Ferrari and you told him one day you were going to buy a gallon of milk at the store. "On sale for $3.99!" you say to him. Now imagine he sped past you on your way to the store and when you arrived there he had bought all the milk at $3.99 and was selling it in the parking lot for $4.01.
You wouldn't necessarily be ruined financially paying $4.01 instead of $3.99. The cost is negligible. But you would probably think he was kind of a jerk.