Speed ​​limit of outgoing PHP requests + curl

Question

Is there a way to limit (with a delay) outgoing PHP + curl requests to an external server, so that there are only n requests per second? PHP is used in Fastcgi mode, so it is impossible to use sleep.

Answer 1

Yes. There's a cutter multiple handler ...

(You can do it in OOP way using this library )

For instance:

// Creates the curl multi handle $mh = curl_multi_init(); $handles = array(); foreach($urls as $url) { // Create a new single curl handle $ch = curl_init(); // Set options curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_TIMEOUT, 300); // Add to the multi handle curl_multi_add_handle($mh,$ch); // Put the handles in an array to loop this later on $handles[] = $ch; } // Execute the multi handle $running=null; do { $mrc = curl_multi_exec($mh,$running); // Added a usleep for 0.50 seconds to reduce load usleep (250000); } while($running > 0); // Get the content of the urls (if there is any) $output = array(); for($i=0; $i<count($handles); $i++) { // Get the content of the handle $content = curl_multi_getcontent($handles[$i]); $output[] = $content; if($printOutput) { echo $content; } // Remove the handle from the multi handle curl_multi_remove_handle($mh,$handles[$i]); } // close the multi curl handle to free system resources curl_multi_close($mh); 

Answer 2

You can control the speed using the token marker algorithm . Since you want to control the speed of all your PHP processes for a single resource, you will need to split the state of the bucket. You can do this in streaming mode with my implementation: bandwidth-throttle / token-bucket

 use bandwidthThrottle\tokenBucket\Rate; use bandwidthThrottle\tokenBucket\TokenBucket; use bandwidthThrottle\tokenBucket\BlockingConsumer; use bandwidthThrottle\tokenBucket\storage\FileStorage; $storage = new FileStorage(__DIR__ . "/api.bucket"); $rate = new Rate(10, Rate::SECOND); $bucket = new TokenBucket(10, $rate, $storage); $consumer = new BlockingConsumer($bucket); $bucket->bootstrap(10); // This blocks all processes to the rate of 10 requests/second $consumer->consume(1); $api->doSomething();