MapReduce Design Patterns: Building Effective Algorithms and Analytics for Hadoop and Other Systems

by Donald Miner, Adam Shook

Until now, design patterns for the MapReduce framework have been scattered among various research papers, blogs, and books. This handy guide brings together a unique collection of valuable MapReduce patterns that will save you time and effort regardless of the domain, language, or development framework you’re using.

Each pattern is explained in context, with pitfalls and caveats clearly identified to help you avoid common design mistakes when modeling your big data architecture. This book also provides a complete overview of MapReduce that explains its origins and implementations, and why design patterns are so important. All code examples are written for Hadoop.

Summarization get a top-level view by summarizing and grouping dataFiltering view data subsets such as records generated from one userData organization reorganize data to work with other systems, or to make MapReduce analysis easierJoin analyze different datasets together to discover interesting relationships piece together several patterns to solve multi-stage problems, or to perform several analytics in the same jobInput and output customize the way you use Hadoop to load or store data"A clear exposition of MapReduce programs for common data processing patterns—this book is indespensible for anyone using Hadoop."--Tom White, author of The Definitive Guide

Genres: Programming, Computer Science, Software, Nonfiction, Technical, Computers, Technology

252 pages, Kindle Edition

First published September 22, 2012

Reviews

[Rinat Sharipov · Rating 5 out of 5]

Big data engineer’s handbook

[Alexei · Rating 4 out of 5]

A very good primer on the bread-and-butter techniques of using Hadoop (also applicable to other MR platforms). Some patterns like filtering are infuriatingly simple, but you'll benefit from knowing the caveats of others, and from charting the MapReduce problem space that these patterns address.

I wish this book covered approximations more, like mentioning that you can do a count distinct query with HyperLogLog, but unfortunately it only ever deals with using Bloom filters for reducing bandwidth requirements of reduce-side joins. Also, something relating to graph processing would be nice to see, because it's such a rich problem domain and knowing how to address it with MapReduce would be very nice.

[Radheshyam · Rating 5 out of 5]

ewrw

[Varun]

grt

[Delhi Irc]

Location: ND6 IRC
Accession No: DL027413

[TN · Rating 4 out of 5]

It is a generally good book. I found most of the patterns well presented. However, a bit more on possible implications and comparisons will be very useful.