Comment by tangoalpha

Pipes was ahead of its time. Built a scraper and aggregator back in 2008-2010, fetching content from over 2000 sources, pumping all of it into a Drupal website. The amount of regex transformations and other things I had in that pipeline - is in hindsight - more complex, larger scale, and more convoluted that any other nifi or airflow pipelines I built later in my career. And all of it was free. What was possible was limited only by one's creativity!