In enterprise environments, I more often see
overuse of message brokers. I am all for targeted use of messaging systems, but what I usually see is an all or nothing approach where as soon as a team starts using messaging they use it for _all_ interprocess communication.
This has some very painful downsides.
First, traditional message brokers (ie queues, topics, persistence, etc) introduce a potential bottleneck for all communication, and on top of that they are _slow_ compared to just using the network (ie HTTP, with no middleman). I've had customers who start breaking apart their monoliths and replacing with microservices, and since the monolith used messaging they use messaging between microservices as well. Well, the brokers were already scaled to capacity for the old system, so going from "1" monolith service to "10" microservices means deploying 10x more messaging infrastructure. Traditional messaging brokers also don't scale all that well horizontally, so that 10x is probably more like 20x.
Second, performance tuning becomes harder for people to understand. A core tenant of "lean" is to reduce the number/length of queues in a system to get more predictable latency and to make it easier to diagnose bottlenecks. But everyone always does the opposite with messaging: massive queues everywhere. By the time the alarms go off, multiple queues are backing up and no one is clear on where the actual bottleneck is.
What I would like to see more of is _strategic_ use of messaging. For example, a work queue to scale out processing, but the workers use HTTP or some other synchronous method to call downstream services. Limiting messaging to very few points in the process still give a lot of the async benefits to the original client, while limiting how often requests hit the broker and making it easier to diagnose performance problems.
This has some very painful downsides.
First, traditional message brokers (ie queues, topics, persistence, etc) introduce a potential bottleneck for all communication, and on top of that they are _slow_ compared to just using the network (ie HTTP, with no middleman). I've had customers who start breaking apart their monoliths and replacing with microservices, and since the monolith used messaging they use messaging between microservices as well. Well, the brokers were already scaled to capacity for the old system, so going from "1" monolith service to "10" microservices means deploying 10x more messaging infrastructure. Traditional messaging brokers also don't scale all that well horizontally, so that 10x is probably more like 20x.
Second, performance tuning becomes harder for people to understand. A core tenant of "lean" is to reduce the number/length of queues in a system to get more predictable latency and to make it easier to diagnose bottlenecks. But everyone always does the opposite with messaging: massive queues everywhere. By the time the alarms go off, multiple queues are backing up and no one is clear on where the actual bottleneck is.
What I would like to see more of is _strategic_ use of messaging. For example, a work queue to scale out processing, but the workers use HTTP or some other synchronous method to call downstream services. Limiting messaging to very few points in the process still give a lot of the async benefits to the original client, while limiting how often requests hit the broker and making it easier to diagnose performance problems.