Sample size is never directly mentioned, but:
> At Triplebyte we evaluate hundreds of engineers every week.
> I pulled the editor / language data for every Triplebyte interview over the last year.
Based on that, if we suppose "hundreds" means "low hundreds," this suggests a sample size of 5K-15K.
That's not a bad sample size for the total population.
But if 35% use Python and only 5% use Ruby, then you're comparing a sample of 1,750 (on the low end) to a sample of 250. It wouldn't surprise me if the Ruby data is more shaky than the Python data, and thus it's not a fair comparison.
Same goes for a lot of the other metrics where the sample sizes vary.
~ Disclaimer: I'm no statistician (but I play one on Correlated.org).
> At Triplebyte we evaluate hundreds of engineers every week.
> I pulled the editor / language data for every Triplebyte interview over the last year.
Based on that, if we suppose "hundreds" means "low hundreds," this suggests a sample size of 5K-15K.
That's not a bad sample size for the total population.
But if 35% use Python and only 5% use Ruby, then you're comparing a sample of 1,750 (on the low end) to a sample of 250. It wouldn't surprise me if the Ruby data is more shaky than the Python data, and thus it's not a fair comparison.
Same goes for a lot of the other metrics where the sample sizes vary.
~ Disclaimer: I'm no statistician (but I play one on Correlated.org).