Troubleshooting PHP and do / while

Question

I have a do / while loop that iterates over database rows. Since it works for many days while processing 100,000 lines, memory consumption is important to keep in check or it crashes. Right now, each iteration adds about 4 kb to the script memory. I use memory_get_usage () to track usage.

I turned off every variable used in the loop, the first thing in every iteration, so I really don't know what else I could do. I assume that do / while collects some data with each iteration, and this is what consumes 4kb of memory. I know that 4kb doesn’t sound that much, but it soon starts to add up when you have 100,000 iterations.

Can someone suggest another way to traverse a large number of database rows or somehow fix this "memory leak"?

edit This shows the UPDATED loop code. Above are just a few require_once () s.

$URLs = new URLs_url(db());
$c = new Curl;
$c->headers = 1;
$c->timeout = 60;
$c->getinfo = true;
$c->follow = 0;
$c->save_cookies = false;

do {
    // Get url that hasn't been checked for a week
    $urls = null;

    // Check week old
    $urls = $URLs->all($where)->limit(10);

    foreach($urls as $url) {
        #echo date("d/m/Y h:i").' | Checking '.$url->url.' | db http_code: '.$url->http_code;

        // Get http code    
        $c->url = $url->url;
        $data = $c->get();

        #echo ' - new http_code: '.$data['http_code'];

        // Save info
        $url->http_code = $data['http_code'];
        $url->lastchecked = time();
        $URLs->save($url);
        $url = null;
        #unset($c);
        $data = null;
        #echo "\n".memory_get_usage().' | ';
        echo "\nInner loop memory usage: ".memory_get_usage();
    }
    echo "\nOuter loop memory usage: ".memory_get_usage();

} while($urls);

Some logs like memory consumption behave in both cycles:

Inner loop memory usage: 611080
Inner loop memory usage: 612452
Inner loop memory usage: 613788
Inner loop memory usage: 615124
Inner loop memory usage: 616460
Inner loop memory usage: 617796
Inner loop memory usage: 619132
Inner loop memory usage: 620500
Inner loop memory usage: 621836
Inner loop memory usage: 623172
Outer loop memory usage: 545240
Inner loop memory usage: 630680
Inner loop memory usage: 632016
Inner loop memory usage: 633352
Inner loop memory usage: 634688
Inner loop memory usage: 636088
Inner loop memory usage: 637424
Inner loop memory usage: 638760
Inner loop memory usage: 640096
Inner loop memory usage: 641432
Inner loop memory usage: 642768
Outer loop memory usage: 556392
Inner loop memory usage: 640416
Inner loop memory usage: 641752
Inner loop memory usage: 643088
Inner loop memory usage: 644424
Inner loop memory usage: 645760
Inner loop memory usage: 647096
Inner loop memory usage: 648432
Inner loop memory usage: 649768
Inner loop memory usage: 651104
Inner loop memory usage: 652568
Outer loop memory usage: 567608
Inner loop memory usage: 645924
Inner loop memory usage: 647260
Inner loop memory usage: 648596
Inner loop memory usage: 649932
Inner loop memory usage: 651268
Inner loop memory usage: 652604
Inner loop memory usage: 653940
Inner loop memory usage: 655276
Inner loop memory usage: 656624
Inner loop memory usage: 657960
Outer loop memory usage: 578732

Answer 1

This bit should probably happen only once before the loop:

$c = new Curl;
$c->headers = 1;
$c->timeout = 60;
...
$c->getinfo = true;
$c->follow = 0;
$c->save_cookies = false;

Edit: Oh, it's all wrapped in a do / while. / Facepalm loop

Edit 2: There is also this important bit:

unset ($ class_object) does not release resources allocated by the object. If used in a loop that create and destroy objects, that can easily lead to a resource problem. explicitly call the destructor to get around the problem.

http://www.php.net/manual/en/function.unset.php#98692

3:

? ?

$URLs = new URLs_url(db());

4:

, .

    $url->http_code = $data['http_code'];
    $url->lastchecked = time();
    $URLs->save($url);

Answer 2

, , .

$c = new Curl, , , unset . unset , ($c, $data) .

Answer 3

, ,

$c = new Curl

Curl , . reset , .

. Unset - , . , (, , Java).

Answer 4

, , 2000 , , cms , , . IIS win xp, - script 60 , , , 2 , , , -.

, script , , , , , . , - ?

Perhaps start it for a certain time before calling yourself, or in your case, maybe check the memory and redirect when the usage is too big?

I used something like this:

Top script:

$started = microtime(true);

Then this is in your loop:

if((microtime(true)-$started) > ($seconds_to_redirect)) {
    //call script with parameter
}

That’s all I can think of.