Tuesday, 4 November 2014

Ignore Some Failed Tasks in MapReduce


In Pig, we can below for MRv1
set mapred.max.map.failures.percent 100
mapred.max.reduce.failures.percent 100
mapred.map.tasks.speculative.execution false

And for MRv2
mapreduce.map.failures.maxpercent 100
mapreduce.reduce.failures.maxpercent 100
mapreduce.map.speculative false


Reference:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html

No comments:

Post a Comment