The problem with that interpretation is that it ignores how much something deviates from the expected random. If everything was completely random:
- 25% of the time, the actual and perceived ranking would be identical.
- 62.5% of the time, the difference would be at most one.
- Only in 37.5% of cases would there be a strong difference in scores (2 or more stars out of four).
Would you say that 44% is a good score on a multiple choice test? Obviously candidates did better than random guessing, but not by enough to call it a strong correlation. Determining the strength of a correlation is what R-squared is for, which makes it the right choice for this analysis.
- 25% of the time, the actual and perceived ranking would be identical.
- 62.5% of the time, the difference would be at most one.
- Only in 37.5% of cases would there be a strong difference in scores (2 or more stars out of four).
Would you say that 44% is a good score on a multiple choice test? Obviously candidates did better than random guessing, but not by enough to call it a strong correlation. Determining the strength of a correlation is what R-squared is for, which makes it the right choice for this analysis.