org.apache.tez.mapreduce.examples

Class GroupByOrderByMRRTest

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.tez.mapreduce.examples.GroupByOrderByMRRTest

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class GroupByOrderByMRRTest
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Simple example that does a GROUP BY ORDER BY in an MRR job Consider a query such as Select DeptName, COUNT(*) as cnt FROM EmployeeTable GROUP BY DeptName ORDER BY cnt; i.e. List all departments with count of employees in each department and ordered based on department's employee count. Requires an Input file containing 2 strings per line in format of For example, use the following: #/bin/bash i=1000000 j=1000 id=0 while [[ "$id" -ne "$i" ]] do id=`expr $id + 1` deptId=`expr $RANDOM % $j + 1` deptName=`echo "ibase=10;obase=16;$deptId" | bc` echo "$id O$deptName" done

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GroupByOrderByMRRTest.MyGroupByReducer Intermediate reducer aggregates the total count per department.
static class	GroupByOrderByMRRTest.MyMapper Mapper takes in a single line as input containing employee name and department name and then emits department name with count of 1
static class	GroupByOrderByMRRTest.MyOrderByNoOpReducer Shuffle ensures ordering based on count of employees per department hence the final reducer is a no-op and just emits the department name with the employee count per department.

Constructor Summary

Constructors

Constructor and Description
GroupByOrderByMRRTest()

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(String[] args)
int	run(String[] args)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

GroupByOrderByMRRTest

public GroupByOrderByMRRTest()

Method Detail

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception