Comment by tr33house

If anyone is starting a new project, I'd recommend looking into Apache Pulsar [0]. It has all the good parts of Kafka with a lot more features useful when scaling

0: https://pulsar.apache.org/