How to efficiently sum two columns in a file with 270,000 + rows in bash

I have two columns in a file and I want to automate the summation of both values โ€‹โ€‹in a row

eg

read write
5    6
read write
10   2
read write
23   44

I then want to summarize the "read" and "write" of each line. In the end, after summing up, I find the maximum amount and put this maximum value in the file. I feel like I need to use grep -v to get rid of the column headers per row, which, as indicated in the answers, makes the code inefficient, since I am preparing the entire file for read-only rows.

I currently have this in a bash script (in a for loop, where $ x is the name of the file) to sum the columns by row

lines=`grep -v READ $x|wc -l | awk '{print $1}'`
line_num=1
arr_num=0


while [ $line_num -le $lines ]
do

    arr[$arr_num]=`grep -v READ $x |  sed $line_num'q;d' | awk '{print $2 + $3}'`
    echo $line_num
    line_num=$[$line_num+1]
    arr_num=$[$arr_num+1]

done

, , 270 000+ . script , . , ?

+4
6

awk:

awk 'BEGIN{print "sum" > "outfile"}NR>1{print $1+$2}' infile >> outfile

:

awk '!(NR%2){print $1+$2}' infile
+5

awk, , , bash :

while read -a line; do      # read each line one-by-one, into an array
                            # use arithmetic expansion to add col 1 and 2
    echo "$(( ${line[0]} + ${line[1]} ))"
done < <(grep -v READ input.txt)

, ( grep), ( grep, ). bash .

<( ), , , while, while. |.

+1

, . , , , . , :

awk '
    NR%2 == 1 {next} 
    NR == 2 {max = $1+$2; next} 
    $1+$2 > max {max = $1+$2}
    END {print max}
' filename
+1

:

awk 'NR==1 { print "sum"; next } { print $1 + $2 }'

, script . (). , , .

Perl Python awk, .

grep, sed awk ; . ; Bash, .

0

, 'header', 'data':

awk '
  BEGIN{ max = 0 }
  {
    if( NR%2 == 0 ){
      sum = $1 + $2;
      if( sum > max ) { max = sum }
    }
  }
  END{ print max }' input.txt

, , :

grep '^[0-9]\+\s\+[0-9]\+$' input.txt | awk '
  BEGIN{ max = 0 }
  {
    sum = $1 + $2;
    if( sum > max ) { max = sum }
  }
  END{ print max }' input.txt
0

, :

grep -v read INFILE | tr -s ' ' + | bc | sort -rn | head -1 > OUTFILE

.

0

Source: https://habr.com/ru/post/1533588/


All Articles