Hadoop MapReduce intermediate output

Is there an output method for registering intermediate (Map Phase) output from a MapReduce job without editing the application? (The application is not mine, but there is a cluster, and I can configure the Hadoop cluster as I want)

+6
source share
1 answer

keep.task.files.pattern can be used to store intermediate files. Intermediate files must be cleaned manually after completion of the task. Since this is a map / reduce task property, it must be set in the configuration file and the jar file is packaged again.

+8
source

Source: https://habr.com/ru/post/899914/


All Articles