I tried adding my request:
set mapred.running.reduce.limit = 25;
AND
set hive.exec.reducers.max = 35;
The latter imprisoned a task with 530 gears up to 35 ..., which makes me think that he is going to try and process 530 gears for shoes at 35.
Now giving
set mapred.tasktracker.reduce.tasks.maximum = 3;
try to find out if this number is some kind of maximum value for a node (previously there were 7 on a cluster with 70 potential reducers).
Update:
set mapred.tasktracker.reduce.tasks.maximum = 3;
There was no effect, it was worth a try.
David source
share