How can I efficiently populate an N x M grid using Perl?

Question

I have a Perl script that parses a data file and writes 5 output files filled with a 1100 x 1300 grid. The script works, but in my opinion it is awkward and probably inefficient. The script is also inherited code, which I modified a bit to make it more readable. However, this is a mess.

Currently, the script reads the data file (~ 4Mb) and puts it in an array. Then it iterates over the array, analyzing its contents and pushing the values ​​into another array and finally printing them to a file in another loop. If the value is not defined for a specific point, it prints 9999. Zero is a valid value.

The data file has 5 different parameters, and each of them is written to its own file.

Sample data:

data for the param: 2
5559
// (x,y) count values
280 40 3  0 0 0 
280 41 4  0 0 0 0 
280 42 5  0 0 0 0 0 
281 43 4  0 0 10 10 
281 44 4  0 0 10 10 
281 45 4  0 0 0 10 
281 46 4  0 0 10 0 
281 47 4  0 0 10 0 
281 48 3  10 10 0 
281 49 2  0 0 
41 50 3  0 0 0 
45 50 3  0 0 0 
280 50 2  0 0 
40 51 8  0 0 0 0 0 0 0 0
...

data for the param: 3
3356
// (x,y) count values

5559 - . : x, y, x- , , , .

, script, , . , . , .

, for-loops?

EDIT:

, , .

1100 x 1300, , . . , x (+ n), y .

UPDATE:

, , , script (~ 3 ). script 50% , , script. , 3- .

script. , . ?

 for my $i (0..$#indata) { # Data file is read to @indata
 ...
   if($indata[$i] =~ /^data for the param:/) { 
     push @block, $i;  #  data borders aka. lines, where block starts and ends
   }
 ...
 }
  # Then handle the data blocks
 for my $k (0..4) {  # 5 parameters
 ...
   if( $k eq '4') {  # Last parameter
     $enddata = $#indata;
   }
   else {
     $enddata = $block[$k+1];
   }
    ...
   for my $p ($block[$k]..$enddata) { # from current block to next block 
    ...
   # Fill data array
    for(my $m=0 ; $m<$n ; $m++){
    $data[$x][$y] = $values[$m];
     }

   }
   print2file();

 }

Answer 1

. 9999 undefined. , , .

#!/usr/bin/perl

use strict; use warnings;
use YAML;

use constant GRID_X => 1100 - 1;
use constant GRID_Y => 1300 - 1;

while (my $data = <DATA> ) {
    if ( $data =~ /^data for the param: (\d)/ ) {
        process_param($1, \*DATA);
    }
}

sub process_param {
    my ($param, $fh) = @_;
    my $lines_to_read = <$fh>;
    my $lines_read = 0;

    $lines_to_read += 0;

    my %data;

    while ( my $data = <$fh> ) {
        next if $data =~ m{^//};
        last unless $data =~ /\S/;
        $lines_read += 1;

        my ($x, $y, $n, @vals) = split ' ', $data;

        for my $i ( 0 .. ($n - 1) ) {
            $data{$x + $i}{$y} = 0 + $vals[$i];
        }
    }
    if ( $lines_read != $lines_to_read ) {
        warn "read $lines_read lines, expected $lines_to_read\n";
    }

    # this is where you would open a $param specific output file
    # and write out the full matrix, instead of printing to STDOUT
    # as I have done. As an improvement, you should probably factor
    # this out to another sub.

    for my $x (0 .. GRID_X) {
        my $row;
        for my $y (0 .. GRID_Y) {
            my $v = 9999;
            if ( exists($data{$x})
                    and exists($data{$x}{$y})
                    and defined($data{$x}{$y}) ) {
                $v = $data{$x}{$y};
            }
            $row .= "$v\t";
        }
        $row =~ s/\t\z/\n/;
        print $row;
    }

    return;
}


__DATA__
data for the param: 2
5559
// (x,y) count values
280 40 3  0 0 0 
280 41 4  0 0 0 0 
280 42 5  0 0 0 0 0 
281 43 4  0 0 10 10 
281 44 4  0 0 10 10 
281 45 4  0 0 0 10 
281 46 4  0 0 10 0 
281 47 4  0 0 10 0 
281 48 3  10 10 0 
281 49 2  0 0 
41 50 3  0 0 0 
45 50 3  0 0 0 
280 50 2  0 0 
40 51 8  0 0 0 0 0 0 0 0

Answer 2

Perl , .

my $matrix = [];
$matrix->[0]->[0] = $valueAt0x0;

,

$matrix = [];
while($ln = <INPUT>) {
  @row = split(/ /, @ln); #assuming input is separated by spaces
  push(@$matrix, \@row);
}
# here you read matrix.  Let print it
foreach my $row (@$matrix) {
  print join(",", @{$row}) . "\n";
}
# now you pruinted your matrix with "," as a separator

, .

Answer 3

, , . . , $_ . flush_output (, " , " ), $matrix[$x][$y] .= ($matrix[$x][$y] ? ',' : '') . join(',', @data); - .

use strict;
use warnings;

my $cur_param;
my @matrix;
while (<DATA>) {
  chomp;
  s/\/\/.*$//;
  next if /^\s*$/;

  if (/^data for the param: (\d+)/) {
    flush_output($cur_param, \@matrix) if defined $cur_param;
    $cur_param = $1;
    @matrix = (); # reset
    # skip the line with number of rows, we're smarter than that
    my $tmp = <DATA>;
    next;
  }

  (my $x, my $y, undef, my @data) = split /\s+/, $_;
  $matrix[$x][$y] ||= [];
  push @{$matrix[$x][$y]}, @data;
}

sub flush_output {
  my $cur_param = shift;
  my $matrix = shift;
  # in reality: open file and dump
  # ... while dumping, do an ||= [9999] for the default...

  # here: simple debug output:
  use Data::Dumper;
  print "\nPARAM $cur_param\n";
  print Dumper $matrix;
}

__DATA__
data for the param: 2
5559
// (x,y) count values
280 40 3  0 0 0 
280 41 4  0 0 0 0 
280 42 5  0 0 0 0 0 
281 43 4  0 0 10 10 
281 44 4  0 0 10 10 
281 45 4  0 0 0 10 
281 46 4  0 0 10 0 
281 47 4  0 0 10 0 
281 48 3  10 10 0 
281 49 2  0 0 
41 50 3  0 0 0 
45 50 3  0 0 0 
280 50 2  0 0 
40 51 8  0 0 0 0 0 0 0 0

data for the param: 3
3356
// (x,y) count values