SQL GROUP BY allows you to perform aggregate functions on data sets; To count all of the stores in each state, to average a series of related numbers, etc. MongoDB has some aggregate functions but they are fairly limited in scope. The MongoDB group function also suffers from the fact that it does not work on sharded configurations. So how do you perform grouped queries using MongoDB? By using MapReduce functions of course (you read the title right?)
Understanding MapReduceUnderstanding MapReduce requires, or at least is made much easier by, understanding functional programming concepts. map and reduce (fold, inject) are functions that come from Lisp and have been inherited by a lot of languages (Scheme, Smalltalk, Ruby, Python).
- A higher-order function which transforms a list by applying a function to each of its elements. Its return value is the transformed list. In MongoDB terms, the map is a function that is run for each Document in a collection and can return a value for that row to be included in the transformed list.
- A higher-order function that iterates an arbitrary function over a data structure and builds up a return value. The reduce function takes the values returned by map and allows you to run a function to manipulate those values in some way.
Some ExamplesLet's start with some sample data:
Lets say I want to count the number of factories in each of the cities (ignore the fact that I could have the same city in more than one state, I don't in my data). For a count, I write a function that "emits" the group by key and a value that you can count. It can be any value, but for simplicity I'll make it 1. emit() is a MongoDB server-side function that you use to identify a value in a row that should be added to the transformed list. If emit() is not called then the values for that row will be excluded from the results.
The next piece is the reduce() function. The reduce function will be passed a key and an array of values that were collected by the map() function. I know my map function returns a 1 for each row keyed by city. So the reduce function will be called with a key "Golden Springs" and a single-element array containing a 1. For "Milwaukee" it will be passed an 4-element array of 1s.
With those 2 functions I can call the mapReduce function to perform my Query.
This results in:
Counting is not the only thing I can do of course. Anything can be returned by the map function including complex JSON objects. In this example I combine the names of all of the Factories in a given City into a simple comma-separated list.
ConclusionThese are fairly simple examples, but I think it helps to work through this kind of simple thing to fully understand a new technique before you have to work with harder examples.
For more on MongoDB check out these books:
- MongoDB: The Definitive Guide
- The Definitive Guide to MongoDB: The NoSQL Database for Cloud and Desktop Computing
- MongoDB for Web Development (Developer's Library)
- How to Design Programs: An Introduction to Programming and Computing - If you want to learn more about functional programming I highly recommend this book.