It's a lot faster if you use python's itertools and operator itemgetter:
def gen_stats(data):
start = time.time()
stats = []
data.sort(key=itemgetter(0))
for p_id, p_items in itertools.groupby(data, key=itemgetter(0)):
num_orders, _, total_quantity, total_price = map(sum, zip(*p_items))
p_id, num_orders, total_quantity = map(int, [p_id, num_orders/p_id, total_quantity])
stats.append([p_id, num_orders, total_quantity, round(float(total_price/num_orders),2)])
end = time.time()
working_time = end-start
return stats, working_time
Results:
Dataset size 55096 records
[[1, 8, 340, 22.55], [2, 4, 94, 15.83], [3, 2, 97, 3.89]]
[[1, 8, 340, 22.55], [2, 4, 94, 15.83], [3, 2, 97, 3.89]]
All results equal: True
Runtime (s)
gen_stats_slow: 28.489468574523926
gen_stats_fast: 0.06000018119812012
