Memory usage increases after each cycle.

Question

I have a foreach loop reading a file containing about 200,000 lines, this is my PHP:

foreach ($this->file as $row) { if ($this->file->valid()) { //init client array $this->initInvoicesArray($row); $this->prepareInvoice(); $row = null; $this->key = $this->file->key(); //add msisdn to msisdn array, and client to clients array self::$MsisdnArray[] = $this->msisdn; self::$InvoicesArray[$this->msisdn] = $this->client; if ($i % 3000 == 0) { //get valid users from table $this->prepareAndSaveValidClients(); } $i++; } } 

and this is the prepareAndSaveValidClients () function:

 public function prepareAndSaveValidClients(){ $query = Doctrine_Query::create() ->select('p.gender, p.email2, u.username, u.first_name, u.last_name, u.email_address, u.is_active, p.msisdn, p.user_id, p.city_id, p.street, p.zipcode, p.msisdn_status') ->from('sfGuardUser u') ->innerJoin('u.Profile as p ON u.id = p.user_id') ->whereIn('p.msisdn', self::$MsisdnArray) ->whereIn('p.status', self::$AllowedStatus); $results = $query->fetchArray(); //instanciat an object collection for payment_notifications $collection = new Doctrine_Collection("payment_notifications"); if (!empty($results)) { foreach ($results as $key => $client) { $invoice = self::$InvoicesArray[$client['Profile']['msisdn']]; $this->initInvoicesArray($invoice); $this->prepareInvoice(); $this->prepareUserProfile($client); $this->prepareClient(); $paymentNotifications = new paymentNotifications(); $paymentNotifications->fromArray($this->client); $collection->add($paymentNotifications); $tel = $client['Profile']['msisdn']; $client = null; } $collection->save(); //clear memory $results = null; $collection = null; self::$MsisdnArray = null; self::$InvoicesArray = null; $this->logSection('tel num', 'added :' . $tel . ' Memory usage : ' . memory_get_usage()); $duration = microtime(true) - $this->startTime; $this->logSection('payment : ', sprintf('added in %s', $duration)); } } 

As for the functions:

  $this->initInvoicesArray($invoice); $this->prepareInvoice(); $this->prepareUserProfile($client); $this->prepareClient(); 

They are only for preparing $this->client

This is the memory usage that is displayed for each cycle:

 >> tel num added :0699946185 Memory usage : 89287596 >> payment : added in 8.6373870372772 >> tel num added :0699983919 Memory usage : 165854544 >> payment : added in 18.373502969742 >> tel num added :0699949623 Memory usage : 241338788 >> payment : added in 29.336947917938 >> tel num added :0699854750 Memory usage : 319173092 >> payment : added in 40.880628824234 

As you can see, I tried to free the memory of these variables:

 $results = null; $collection = null; self::$MsisdnArray = null; self::$InvoicesArray = null; 

But in vain, memory usage continues to increase after each cycle, which leads to the fatal error An Allowed Memory size exausted . How can I optimize it?

Thanks.

Answer 1

I solved the problem 2 months ago and Memory_usage became stable, but I forgot to post the solution here.

What I did was I removed Doctrine, and I used mere PDO and Sql for the select query .

Answer 2

Doctrine objects have circular links that php has freedom issues. To solve this problem, doctrine provides a free() method, call it in the collection just before setting the value to zero:

 $collection->free(true); $collection = null; 

Answer 3

After calling $this->prepareAndSaveValidClients(); you can also use the gc_collect_cycles() function. This will help with possible memory leaks.