Comment

Comment by zmmmmm

> I would appreciate if you elaborated on separating step definitions from dependency definitions

As I said, I only very quickly skimmed since I'm busy, I might have overlooked information, and apologies in that case. But take the example from the front page:

    evergreens.csv <- contracts.csv
      grep Evergreen $INPUT > $OUTPUT
So now suppose a new requirement comes along - Evergreen is also called "Neverbrown" sometimes. It's decided the best way is to convert all references at input so nothing else gets confused downstream. So I need an extra step, now

    renamed.csv <- contracts.csv:
        sed 's/Neverbrown/Evergreen/g' $INPUT > $OUTPUT

    evergreens.csv <- renamed.csv:
        grep Evergreen $INPUT > $OUTPUT
Adding this step forced me to modify the declaration of the original command, even though what I added had nothing to do with that command. With Bpipe, for example, you say

    extract_evergreens = { 
      exec "grep Evergreen $input > $output" 
    }

    fix_names = { 
      exec "sed 's/Neverbrown/Evergreen/g' $input > $output"
    }
Then you define your pipeline order separately -

    run { fix_names + extract_evergreens }
If I get contracts from a different source that don't need the renaming, I can still run my old version and I'm not changing the definition of anything:

    run { extract_evergreens }
Hope this explains what I mean, and again apologies if this is all clearly explained in your docs and I just jumped to conclusions from the simple examples!
parent zmmmmm 13 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton