I've been playing around with the Aggregation Framework lately (using the release candidate). The performance seems to be pretty reasonable, especially when compared to similar tasks with the old MR framework. A quick and dirty benchmark number in case anyone is interested:
* Obligatory unscientific, probably not meaningful, etc. disclaimer.
Mongo Version: 2.2.0-rc1
Hardware: MBP, Snow Leapord, 2.2 GHz Intel Core i7, 8 GB mem
Data: Single collection with 500k records (machine generated time-series event data)
Query Pipeline:
[
{
$match: { ts: { $gte: 1293858000000, $lt: 1296536400000 } }
},
{
$group: {
_id: 'aggregations',
sum: { $sum: '$foo' },
num: { $sum: 1 },
avg: { $avg: '$bar' }
}
}
]
Results: The time range matched against above matches 42,466 documents within the collection. The average response time over 50 runs is 419ms. Not exactly "Big Data OLAP" stuff just yet, but plenty fast enough for most use cases involving reasonably small sets of data. Great job to the MongoDB team!
* Obligatory unscientific, probably not meaningful, etc. disclaimer.
Mongo Version: 2.2.0-rc1
Hardware: MBP, Snow Leapord, 2.2 GHz Intel Core i7, 8 GB mem
Data: Single collection with 500k records (machine generated time-series event data)
Query Pipeline:
Results: The time range matched against above matches 42,466 documents within the collection. The average response time over 50 runs is 419ms. Not exactly "Big Data OLAP" stuff just yet, but plenty fast enough for most use cases involving reasonably small sets of data. Great job to the MongoDB team!