CarolineW

Wow, I would never have expected my comments to have been interpreted that way. Genuinely shocked.

Thank you for the feedback.

Actually, we are pretty marginal on sed, but point taken. It feels like there's a difference between "stream mode" and "program mode".

I remember when awk was first implemented. sed was already standard, and awk was this new thing. I love it, and for some things it's my "go to" language. That colors how I think of it - I think of it as a language.

But this has been done to death, everyone is jumping on me, so there seems little to add.

OK, so as kazinator has pointed out, awk is now a mandatory part of Posix, and so is genuinely a part of "shell". My reply there says that I and my colleagues still think of awk as fundamentally different from other pipeline facilities such as tr, sed, sort, uniq, and so on, but I can see why it could, perhaps should, be though of as being "shell".

So it's shell. I might, however, given my background, and remembering as I do its first introduction, always have trouble thinking of it as such.

That I did not know - thank you. We here still think of awk as fundamentally different from other pipeline facilities such as tr, sed, sort, uniq, and so on, but I can see why it could, perhaps should, be though of as being "shell".

I guess I was triggered by the fact that the proposed shell solution is:

* not on a command line (although it could be),

* is significantly longer than the original command line solution, and

* gives a different result.

But you're right, it's shell. I might, however, given my background, and remembering as I do its first introduction, always have trouble thinking of it as such.

It's not a cop-out, we disagree.

> Nor is finding "common ground" the goal (this is not some sort of compromise between political parties, it's a technical argument).

We disagree. When there is a disagreement, finding what you agree with the the first step in finding where the lines of reasoning diverge. Finding common ground is the first step in resolving differences.

> Getting things right is the goal.

Sometimes in software there are judgement calls. Maybe this is one of them, maybe our definitions differ. Sometimes definitions differ because of context or experience. In each case, the terms used are not right or wrong, they are definitions that are useful in the context.

> For which, sometimes one party or the other may have to admit they are wrong - including me.

This is not an "I'm right, you're wrong" situation. By my experience, in my context, what you wrote would be called a "shell solution" in the same sense as the original command-line solution would be called a "shell solution."

You think that invoking AWK from the command line means that it's still a command-line script. Your definition of the terms means that you accept that invoking AWK still lets you call it a "shell solution."

I think that is fundamentally and structurally different from using command line utilities such as tr, sed, sort, and uniq.

So my position is clear - your solution that you call "shell" is not, in my opinion, just "shell". To me, your solution is an AWK solution, and you feed the output from your AWK program through shell utilities.

You are using the terms in a manner that is different from how I'm using them, that much is now clear.

Do you agree that you have written a shell script that invokes a program written in AWK?

Would it be different if you wrote a shell script that invoked a C program by calling a C interpreter? Would you still call it a "shell solution to the problem?"

Does it matter? Really? I've made clear why I've said that I don't class your solution as being shell, why do you care?

Then we'll agree to disagree. I think you are wrong on so many points here, it's clear we're not going to agree, and probably won't find common ground.

Thank you, by the way, for your references to various published material. FWIW, I've worked with BCPL, C, AWK, C++, Unix, Linux, GNU, and much, much more, for the last four decades or so, so I'm not inexperienced, and I have read most of the classics. That also doesn't mean I'm right, but it does mean that I have a basis for my opinions.

So thank you for your offer to school me, but I'll decline, and, as I say, accept that we disagree.

I read your entire post, and I didn't miss any of the points you make. I simply disagree with you.

Quoting from your post:

    And here is my initial solution in UNIX shell:

    # bentley_knuth.sh

    # Usage:
    # ./bentley_knuth.sh n file
    # where "n" is the number of most frequent words 
    # you want to find in "file".
    
    awk '
        {
            for (i = 1; i <= NF; i++)
                word_freq[$i]++
        }
    END     {
                for (i in word_freq)
                    print i, word_freq[i]
            }
    ' < $2 | sort -nr +1 | sed $1q

So you invoke awk, and then run the output of awk through sort and sed.

You're doing all the word counting in awk.

Yes, you're invoking awk from a shell script, but that's really not the same thing as "using shell." McIlroy’s solution is genuinely shell:

    tr -cs A-Za-z '
    ' |
    tr A-Z a-z |
    sort |
    uniq -c |
    sort -rn |
    sed ${1}q

"awk" is generally accepted as a full programming language, whereas "tr", "sort", "uniq", and "sed" are command line utilities. I don't think "awk" classes as a command line utility, so I don't class your solution as "shell".

Perhaps you don't agree, perhaps you think "awk" is a command line utility. If so, then we'll agree to disagree.