I can't say this trend surprises me. Years back I was an IETF meeting when people were already mentioning that the old saying "IP over everything, everything over IP" was slowly being phase out for "HTTP over everything, everything over HTTP".
The thing to realize here is how much middleboxes sit between a client and a server, boxes which are aware of higher layers than just IP. It's impossible today to deploy an Internet-wide layer-4 protocol other than TCP or UDP, and increasingly it will be harder and harder to deploy a truly universal L7 protocol but HTTP. That's my limited view of this issue, how things like IPv6 might affect it I have no idea.
The thing to realize here is how much middleboxes sit between a client and a server, boxes which are aware of higher layers than just IP. It's impossible today to deploy an Internet-wide layer-4 protocol other than TCP or UDP, and increasingly it will be harder and harder to deploy a truly universal L7 protocol but HTTP. That's my limited view of this issue, how things like IPv6 might affect it I have no idea.