Reducer, Mapreduce, does not show an error, but does not give the desired result

Sales Driver Class

package mr.map;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.FloatWritable;
//import org.apache.hadoop.mapreduce.Mapper;
//import org.apache.hadoop.mapreduce.Reducer;

public class SalesDriver 
{
    public static void main(String args[]) throws Exception
    {
        Configuration c=new Configuration();
        Job j=new Job(c,"Sales");

        j.setJarByClass(SalesDriver.class);
        j.setMapperClass(SalesMapper.class);
        j.setReducerClass(SalesReducer.class);

        //j.setNumReduceTasks(0);
        j.setOutputKeyClass(Text.class);
        j.setOutputValueClass(FloatWritable.class);

        Path in=new Path(args[0]);
        Path out=new Path(args[1]);

        FileInputFormat.addInputPath(j, in);
        FileOutputFormat.setOutputPath(j, out);

        System.exit(j.waitForCompletion(true)?0:1);
    }
}

Sales Match Class

package mr.map;

import java.io.IOException;

//import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.FloatWritable;
import org.apache.hadoop.io.LongWritable;
//import org.apache.hadoop.io.DoubleWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

public class SalesMapper extends Mapper<LongWritable, Text, Text, FloatWritable>
{
    public void map(LongWritable k, Text v, Context con) throws IOException, InterruptedException
    {
        String w[]=v.toString().split(" ");
        String product=w[3];
        //String store=w[2];
        //float cost=Integer.parseInt(w[4]);
        float costx = Float.parseFloat(w[4]);

        //String newline= product+","+store; //","+costx;
        //String newline = product;
        con.write(new Text(product), new FloatWritable(costx));
    }
}

Sales Reducer Class

package mr.map;

import java.io.IOException;

import org.apache.hadoop.io.FloatWritable;
//import org.apache.hadoop.io.IntWritable;
//import org.apache.hadoop.io.DoubleWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

public class SalesReducer extends Reducer<Text, FloatWritable, Text, FloatWritable>
{
    public void reduce(Text k, Iterable<FloatWritable>vlist, Context con) throws IOException, InterruptedException
    {
        int tot=0;
        for (FloatWritable v:vlist)
        {
            tot += v.get();
        }
        //int total= (int)tot;
        con.write(new Text(k), new FloatWritable(tot));
    }
}

MapReduce Result

MapReduce Result

I can’t understand why the whole result comes out in large floating point numbers and around the number 5.7480884E7.

The following is an example of input to mapreduce:

  2012-01-01 09:00 San Jose Men Clothing 214.05 Amex              
  2012-01-01 09:00 Fort Worth Women Clothing 153.57 Visa          
  2012-01-01 09:00 San Diego Music 66.08 Cash                       
  2012-01-01 09:00 Pittsburgh Pet Supplies 493.51 Discover          
  2012-01-01 09:00 Omaha Children Clothing 235.63 MasterCard      
  2012-01-01 09:00 Stockton Men Clothing 247.18 MasterCard        
  2012-01-01 09:00 Austin Cameras 379.6 Visa                        
  2012-01-01 09:00 New York Consumer Electronics 296.8 Cash         
  2012-01-01 09:00 Corpus Christi Toys 25.38 Discover               
  2012-01-01 09:00 Fort Worth Toys 213.88 Visa                      
  2012-01-01 09:00 Las Vegas Video Games 53.26 Visa                 
  2012-01-01 09:00 Newark Video Games 39.75 Cash                    
  2012-01-01 09:00 Austin Cameras 469.63 MasterCard                 
  2012-01-01 09:00 Greensboro DVDs 290.82 MasterCard                
  2012-01-01 09:00 San Francisco Music 260.65 Discover              
  2012-01-01 09:00 Lincoln Garden 136.9 Visa                        
  2012-01-01 09:00 Buffalo Women Clothing 483.82 Visa             
  2012-01-01 09:00 San Jose Women Clothing 215.82 Cash            
  2012-01-01 09:00 Boston Cameras 418.94 Amex                       
  2012-01-01 09:00 Houston Baby 309.16 Visa                         
  2012-01-01 09:00 Las Vegas Books 93.39 Visa                       
  2012-01-01 09:00 Virginia Beach Children Clothing 376.11 Amex   
  2012-01-01 09:01 Riverside Consumer Electronics 252.88 Cash       
  2012-01-01 09:01 Tulsa Baby 205.06 Visa                           
  2012-01-01 09:01 Reno Crafts 88.25 Visa                           
  2012-01-01 09:01 Chicago Books 31.08 Cash                         
  2012-01-01 09:01 Fort Wayne Men Clothing 370.55 Amex            
  2012-01-01 09:01 San Bernardino Consumer Electronics 170.2 Cash   
  2012-01-01 09:01 Madison Men Clothing 16.78 Visa                
  2012-01-01 09:01 Austin Sporting Goods 327.75 Discover            
  2012-01-01 09:01 Portland CDs 108.69 Amex                         
  2012-01-01 09:01 Riverside Sporting Goods 15.41 Discover          
  2012-01-01 09:01 Reno Toys 80.46 Visa                             
  2012-01-01 09:01 Anchorage Music 298.86 MasterCard    
+4
source share
2 answers

Change the output value of the gearbox to text and convert Float to a string in the expected format.

String.format("%f",tot)

Follow the posts for more details. Number formatting:
with scientific
without scientific notation

Gearbox:

public class SalesReducer extends Reducer<Text, FloatWritable, Text, Text>
{
    public void reduce(Text k, Iterable<FloatWritable>vlist, Context con) throws IOException, InterruptedException
    {
        float tot=0;
        for (FloatWritable v:vlist)
        {
            tot += v.get();
        }
        //int total= (int)tot;
        con.write(new Text(k), new Text(String.format("%f",tot)));
    }
}
0
source

int.
, -, int float .
-, , int.

tot int float double.

double tot=0;
0

Source: https://habr.com/ru/post/1651447/


All Articles