Comment by ameyamk

At LinkedIn we heavily use Azkaban for this. (Open source: https://azkaban.github.io/) Azkaban API can be used to launch offline computation jobs as necessary - Azkaban ensures monitoring, SLA alerting, failed restarts and other dependency management etc.