I am having trouble finding additional information about a particular curl pen in a multi curl scenario. Here is the code.
$job_count = 5;
while ( $eachPr = $prList->fetch () ) {
for ( $job_number = 1 ;
$job_number <= $job_count ;
$job_number ++ , $index ++ ) {
$url = $this->getURL ( $eachPr[ "name" ] ,
$eachPr[ "category" ] ) ;
$this->log ( $url ) ;
$curl_handle = curl_init () ;
curl_setopt ( $curl_handle ,
CURLOPT_USERAGENT ,
$userAgent ) ;
curl_setopt ( $curl_handle ,
CURLOPT_URL ,
$url ) ;
curl_setopt ( $curl_handle ,
CURLOPT_FAILONERROR ,
TRUE ) ;
curl_setopt ( $curl_handle ,
CURLOPT_FOLLOWLOCATION ,
TRUE ) ;
curl_setopt ( $curl_handle ,
CURLOPT_AUTOREFERER ,
TRUE ) ;
curl_setopt ( $curl_handle ,
CURLOPT_RETURNTRANSFER ,
TRUE ) ;
curl_setopt ( $curl_handle ,
CURLOPT_COOKIE ,
$cookie ) ;
var_dump($curl_handle);
curl_multi_add_handle ( $multi_handler ,
$curl_handle ) ;
$eachPr = $prList->fetch () ;
}
do {
while ( ($execrun = curl_multi_exec ( $multi_handler ,
$running )) == CURLM_CALL_MULTI_PERFORM ) ;
if ( $execrun != CURLM_OK ) {
break ;
}
while ( $done = curl_multi_info_read ( $multi_handler ) ) {
$info = curl_getinfo ( $done[ 'handle' ] ) ;
$output = curl_multi_getcontent ( $done[ 'handle' ] ) ;
var_dump($info);
curl_multi_remove_handle ( $multi_handler ,
$done[ 'handle' ] ) ;
}
if ( $running ) {
curl_multi_select ( $multi_handler ,
30 ) ;
}
} while ( $running ) ;
file_put_contents ( $symbols_index_file ,
$index ) ;
$sleep_interval = rand ( 5 ,
10 ) ;
$this->log ( " Sleeping Now For " . $sleep_interval . " seconds" ) ;
sleep ( $sleep_interval ) ;
$index ++ ;
}
curl_multi_close ( $multi_handler ) ;
So, here we iterate over the list of the 11K product with while ( $eachPr = $prList->fetch () ). Then, taking 5 products at a time, I initialize the curl handles, which I add to the multi handle.
Handles are executed in a do while loop. Here the problem arises after selecting a request that was made only using $done = curl_multi_info_read ( $multi_handler ). Each response is passed to another thread that processes another task. And for each stream, a product name, a product identifier, and a raw html response are required. Here's how each stacker is initialized.
$this->work_pool[] = $this->submit ( new PrStacker ( $eachPr[ "name" ] ,
$eachPr[ "id" ] ,
$output ) ) ;
, . , PrStacker, , , , . , .
, / curl , , , . , .
, , , .