Thursday, August 1, 2013

Hadoop Chain Mapper

Sometimes before the reducer could actually take on its job- there may be a actual need to process the actual data to make it more suitable.

 
To illustrate this we want the Mapper to operate in phases.
Let us say we consider two phases.
First phase does the cleaning up task for removing the unwanted data.
The second phase does the actual mapping task - the output of the final second phase mapper is passed to the reducer.


Thus we could chain several mappers and indicate the hadoop system to work accordingly.

No comments:

Post a Comment