Comment by serial_dev

I would have had the same question with Apache Storm, it sounds to me that these tools would solve the described problem relatively well (and now that I think about it, Spark even has Python support).