Capturing output with Perl until a specific pattern is found

Question

It seems to me that I am missing something very simple here, but this is the first time I needed to do this, and it is difficult for me to find an example.

I have a giant foreach that scans output logs and extracts various bits of information based on regular expression matching. My main problem is that several larger output types have headers and footers, like *** Begin bangle tracking log*** , followed by a few gibberish lines, and then ***End bangle tracking log*** .

Is there a way from the foreach have an inner loop that stores all the lines until a footer is found?

 foreach my $line( @parseme ) { if( $line =~ m/***Begin bangle tracking log***/ ) { #Help! Push all lines into an array until bangle tracking footer is found. } if( $line =~ m/Other stuff I am tracking/ ) { #Do other things } } 

Answer 1

You can use the range operator, which acts as a trigger in a scalar context:

 foreach ( @parseme ) { if ( /Begin bangle tracking log/ .. /End bangle tracking log/ ) { push @array, $_; } # other stuff... } 

I used $_ for the foreach because it allows for a more concise syntax. You can use another variable if you want, but then you have to write a condition something like:

 if ( $line =~ /Begin .../ .. $line =~ /End .../ ) { 

which may be more readable in some additional parentheses:

 if ( ($line =~ /Begin .../) .. ($line =~ /End .../) ) { 

One problem with the trigger operator is that it remembers its state even after the loop ends. This means that if you intend to start the loop again, you really need to make sure that the @parseme array ends with the line corresponding to the line /End .../ regexp, so that the trigger will be in a known state the next time the loop starts.

Edit:. If you want to process the collected lines as soon as you reach the footer line, you can do this by specifying the return value of the statement .. which ends with E0 on the last line:

 foreach ( @parseme ) { my $in_block = /Begin bangle tracking log/ .. /End bangle tracking log/; if ( $in_block ) { push @array, $_; } if ( $in_block =~ /E0$/ ) { # last line # process the lines in @array @array = (); } # other stuff... } 

Answer 2

You can do this somewhat easily by running a primitive state machine:

 my $inside_bangle = 0; # 0=outside block, 1=inside my @buffer; foreach my $line( @parseme ) { if ($line =~ m/***Begin bangle tracking log***/ ) { $inside_bangle = 1; next; } if ($line =~ m/***End bangle tracking log***/ ) { $inside_bangle = 0; # PROCESS @buffer somehow next; } if ($inside_bangle) { push @buffer, $line; next; } if ($line =~ m/other stuff i am tracking/ ) { #Do other things } } 

Another option is to use a trigger ( .. )

Answer 3

You are probably looking for an operator .. that has some magic properties when applied with regular expressions. The following example is stolen from PLEAC :

 while (<>) { if (/BEGIN PATTERN/ .. /END PATTERN/) { # line falls between BEGIN and END in the # text, inclusive. } } 

Inside the block, add an array variable of your choice.

Answer 4

And now you can store multiple instances of your fragment of the log if this happens more than once (original DVK code):

 my $inside_bangle = 0; # 0=outside block, 1=inside my %buffer; my $index = 0; foreach my $line( @parseme ) { if ($line =~ m/***Begin bangle tracking log***/ ) { $inside_bangle = 1; next; } if ($line =~ m/***End bangle tracking log***/ ) { $inside_bangle = 0; $index++; # PROCESS @buffer somehow next; } if ($inside_bangle) { push @{ $buffer{$index} }, $line; next; } if ($line =~ m/other stuff i am tracking/ ) { #Do other things } } 

Here is what I wrote initially, but I thought the DVK code was more readable and tidy:

 open FILE, "<", 'testfile.log'; @parseme = <FILE>; my $initialize = shift @parseme; my $startLogging = $initialize =~ m/^Log Start$/ ? 1 : 0; # test if first line of array is start of log my %storage = (); my $index = 0; foreach my $line (@parseme) { $startLogging = 1 if $line =~ m/^Log Start$/; if ($startLogging == 1) { push( @{ $storage{$index} }, $line ) unless $line =~ m/(Log Start|Log End)$/; if ($line =~ m/^Log End$/) { $startLogging = 0; $index++; } } }