Skip objects when they are duplicated, when they are duplicated

Question

I play a little with Symfony2 and Doctrine2.

I have an Entity that has a unique title, for example:

class listItem { /** * @orm:Id * @orm:Column(type="integer") * @orm:GeneratedValue(strategy="AUTO") */ protected $id; /** * @orm:Column(type="string", length="255", unique="true") * @assert:NotBlank() */ protected $title; 

now I get json and update my database with these elements:

 $em = $this->get('doctrine.orm.entity_manager'); foreach($json->value->items as $item) { $listItem = new ListItem(); $listItem->setTitle($item->title); $em->persist($listItem); } $em->flush(); 

works great for the first time. but the second time I get a sql error (of course): Integrity constraint violation: 1062 Duplicate entry

sometimes my json file is updated, and some of them are new, some are not. Is there a way to tell the entity manager to skip duplicate files and just insert new ones?

What is the best way to do this?

Thanks for the help. Please leave a comment if something is unclear

Edit:

what works for me does something like this:

 $uniqueness = $em->getRepository('ListItem')->checkUniqueness($item->title); if(false == $uniqueness) { continue; } $listItem = new ListItem(); $listItem->setTitle($item->title); $em->persist($listItem); $em->flush(); } 

checkUniqueness is a method in my ListItem registry that checks if the header is already in my db.

This is terrible. These are 2 database queries for each item. it ends with about 85 database queries for this action.

Answer 1

How to get all current headers in an array first and check the insertion of a header for current headers in this array

 $existingTitles = $em->getRepository('ListItem')->getCurrentTitles(); foreach($json->value->items as $item) { if (!in_array($item->title, $existingTitles)) { $listItem = new ListItem(); $listItem->setTitle($item->title); $em->persist($listItem); } } $em->flush(); 

getCurrentTitles () needs to be added to the ListItem Repo to simply return an array of headers.

This requires only one additional database query, but it costs more in memory to store the current headers in the array. There may be problems with this method if your dataset for ListItem is very large.

If the number of elements you want to insert each time is not too large, you can change the getCurrentTitles () function to request all these elements with the headers you are trying to insert. Thus, the maximum amount of $ existingTiles that you will return will be the size of your insert data list. You can then complete your checks as described above.

 // getCurrentTitles() - $newTitles is array of all new titles you want to insert return $qb->select('title') ->from('Table', 't') ->in('t.title = ', $newTitles) ->getArrayResult(); 

Answer 2

If you use an entity that already exists in the manager, you need to combine it.

Here is what I would do (not tested yet):

 $repository = $this->get('doctrine.orm.entity_manager'); foreach($json->value->items as $item) { $listItem = new ListItem(); $listItem->setTitle($item->title); $em->merge($listItem); // return a managed entity // no need to persist as long as the entity is now managed } $em->flush();