I use Scalding to process records with many (> 22) fields. At the end of the process, I would like to write the names of the final pipe names to a file. I know this is possible because the Mapper and Reducer magazines show this information. I would like to receive this information in the work itself, in order to use it as the basis for the scheme of the poor. If this is not possible, then is there a good way to use the Pipes API safe for large records (i.e. without resorting to arbitrarily nested tuples or class classes)?
source
share