Is there a way to find out which objects and how many of them I have in my memory?

Question

I have a php script that uses Doctrine2 and Zend to compute some things from a database and send some emails to 30,000 users.

My script is a memory leak, and I want to know what objects are used by this memory, and if possible, who keeps a reference to them (thus preventing them from being freed).

Im using php 5.3.x, so simple circular link problems should not be a problem.

Ive tried using the xdebug trace capabilities to get mem_delta without success (too much data).

Ive tried to manually add memory_get_usage before and after important functions. But the only conclusion that I received was that I lost about 400 thousand. Per user and 3000 users, which gives me 1 GB, which I have.

Are there other ways to find out where and why a memory leak occurs? Thanks

Answer 1

You can try sending 10 letters and then paste this

get_defined_vars(); 

http://nz.php.net/manual/en/function.get-defined-vars.php

At the end of the script or after sending the email (depending on how your code is configured).

This should tell you what else is uploaded and what you can disable / turn into a link.

In addition, if two loaded things are loaded, you will get this near the beginning and end of your code and determine the difference.

Answer 2

30,000 hydration sites are plentiful. Doctrine 2 is stable, but there are some bugs, so I'm not too surprised by the memory leak problems.

Although with smaller datasets, I had some good success using the doctrine of batch processing and creating an iterative result.

You can use the code from the examples and add gc_collect_cycles() after each iteration. You should test it, but for me, the batch size of about 100 or so worked fine - this number gave a good balance between performance and memory usage.

It is very important that the script recognizes which objects are being processed so that it can be reloaded without any problems and resume normal operation without sending emails twice.

 $batchSize = 20; $i = 0; $q = $em->createQuery('select u from MyProject\Model\User u'); $iterableResult = $q->iterate(); while (($row = $iterableResult->next()) !== false) { $entity = $row[0]; // do stuff with $entity here // mark entity as processed if (($i % $batchSize) == 0) { $em->flush(); $em->clear(); gc_collect_cycles(); } ++$i; } 

Anyway, maybe you should rethink your architecture for this script a bit, since ORM is not suitable for processing large pieces of data. Maybe you can get away from working with raw SQL strings?

Answer 3

This is not a tool that will give you what you need, but perhaps it will help you. If you have not already done so, you can implement an identification card template, where each time you create an object, it is registered using an identification card, so at any time you can call the chat and see which objects are downloaded or say to upload any loaded objects.

http://martinfowler.com/eaaCatalog/identityMap.html