Comment

Comment by kod

So the detailed post on datasets at https://databricks.com/blog/2016/01/04/introducing-spark-dat...

uses groupBy

I'm pretty sure based on previous comments you've made that groupBy was one of the things you'd rather eliminate from the RDD api, because of the performance impact compared to reduceByKey (which is almost always what people should be using instead).

Are you at all worried about confusion if groupBy now performs ok on datasets, but not on rdds?

parent kod 11 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton