How to extract data from Parallel :: ForkManager in perl

Question

I use Google only to explain my problem.
I have calculated the wait time for a google search request and I will print it.
Now I need data such as maximum time, minimum time, average time. Therefore, I am trying to move the data to an array that can be subsequently manipulated. The array comes out empty. This seems to be a parallel-forkmanager limitation. Please suggest some work.

My code is:

use Parallel::ForkManager; use WWW::Mechanize; use LWP::UserAgent; use Time::HiRes qw/gettimeofday/; use Time::Format qw/%time/; use POSIX qw( strftime ); use Time::HiRes qw( gettimeofday ); $count=5; @arr=(); $pm = new Parallel::ForkManager($count); for(1..$count) { $pm->start and next; $m = WWW::Mechanize->new(); $m->get( "http://www.google.com" ); $s1=gettimeofday; my ($secs, $microsecs) = gettimeofday(); print strftime("%H:%M:%S", localtime($secs)) . sprintf(".%04d", $microsecs/10); $m->submit_form( form_number => 1, fields => {q=>'abcd'}, ); print " "; print $m->title; print " "; $s2=gettimeofday; my ($secs, $microsecs) = gettimeofday(); print strftime("%H:%M:%S", localtime($secs)) . sprintf(".%04d", $microsecs/10); $s3=$s2-$s1; $s3=$s3*1000; print " $s3\n"; push(@arr,$s3); $pm->finish } $pm->wait_all_children; ## wait for the child processes foreach(@arr) { print "$_\n"; } 

Answer 1

You need to add run_on_finish callback to the manager instance and call finish method with 2 arguments to the child element. Thus, you can get the second parameter to the finish call as the sixth argument to the run_on_finish .

 # parent my @arr; $pm->run_on_finish(sub { push(@arr, ${$_[5]}); }); # child my $val = 42; $pm->finish(0, \$val); # parent $pm->wait_all_children(); print($_, "\n") for @arr; 

Answer 2

This is not so much a limitation of Parallel::ForkManager as the general limit of parallel code. fork spawns a new process that is a copy of the existing one. It has its own memory space, and everything that changes between a child and a parent is ... well, changed.

Threading has the same problem.

There is a rather large chapter on how to work with interprocess communication - perlipc

Parallel::ForkManager , in particular, has several mechanisms for returning data to the parent process. Personally - I really disagree with this, because while it is working, it is also trying to be a more general solution to the problem.

I would rather think about how to use channels to pass information back to the parent process.

 use strict; use warnings; use Data::Dumper; use Parallel::ForkManager; use IO::Pipe; my $pm = Parallel::ForkManager->new(2); for ( 1 .. 4 ) { my $pipe = IO::Pipe->new(); my $pid = $pm->start; if ($pid) { #parent $pipe->reader->autoflush; while (<$pipe>) { print "Got data from $pid: $_\n"; } close ( $pipe ); } else { #is child $pipe->writer->autoflush; print {$pipe} "Child $$ says hello!"; close($pipe); print "Child $$ exiting\n"; } $pm->finish; }