- MapReduce 2.0 (MRv2): CDH 5 includes MapReduce 2.0 (MRv2) running on YARN. The fundamental idea of the YARN architecture is to split up the two primary responsibilities of the JobTracker — resource management and job scheduling/monitoring — into separate daemons: a global ResourceManager (RM) and per-application ApplicationMasters (AM). With MRv2, the ResourceManager (RM) and per-node NodeManagers (NM), form the data-computation framework. The ResourceManager service effectively replaces the functions of the JobTracker, and NodeManagers run on slave nodes instead of TaskTracker daemons. The per-application ApplicationMaster is, in effect, a framework-specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks. For details of the new architecture, see Apache Hadoop NextGen MapReduce (YARN).
Thursday 26 June 2014
One Sentence to Explain MR v2
Labels:
CDH
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment