Comment Comment by ora600 According to the article, Yelp had 7 different data sources and similar number of targets.If they wrote a loader for each combination, they'd end up with 49 combinations. Not to mention 7 loaders to write every time they add an app.With Kafka - they just need to connect each thing to Kafka - 14 connectors instead of 49.This is pretty much the scenario Kafka was invented for, and you get stream processing for free: https://engineering.linkedin.com/distributed-systems/log-wha... parent ora600 10 years ago
If they wrote a loader for each combination, they'd end up with 49 combinations. Not to mention 7 loaders to write every time they add an app.
With Kafka - they just need to connect each thing to Kafka - 14 connectors instead of 49.
This is pretty much the scenario Kafka was invented for, and you get stream processing for free: https://engineering.linkedin.com/distributed-systems/log-wha...