Is Django - SQL bulk get_or_create possible?

Question

I use get_or_create to insert objects into the database, but the problem is that running 1000 takes too much time at once.

I tried bulk_create , but it does not provide me with the necessary functionality (creates duplicates, ignores a unique value, does not cause the post_save signals that I need).

Is it even possible to do get_or_create in bulk with a custom sql query?

Here is my sample code:

related_data = json.loads(urllib2.urlopen(final_url).read())

for item in related_data:

    kw = item['keyword']
    e, c = KW.objects.get_or_create(KWuser=kw, author=author)
    e.project.add(id)
    #Add m2m to parent project

related_data contains 1000 lines that look like this:

[{"cmp":0,"ams":3350000,"cpc":0.71,"keyword":"apple."},
{"cmp":0.01,"ams":3350000,"cpc":1.54,"keyword":"apple -10810"}......]

The KW model also sends a signal, which I use to create another parent model:

@receiver(post_save, sender=KW)
def grepw(sender, **kwargs):
    if kwargs.get('created', False):
        id = kwargs['instance'].id
        kww = kwargs['instance'].KWuser
        # KeyO 
        a, b = KeyO.objects.get_or_create(defaults={'keyword': kww}, keyword__iexact=kww)
        KW.objects.filter(id=id).update(KWF=a.id)

, , , , ?

Answer 1

:

fooobar.com/questions/1335523/...

, commit_on_success, . . . :

transactions

from django.db import transaction

@transaction.atomic
def lot_of_saves(queryset):
    for item in queryset:
        modify_item(item)
        item.save()

Answer 2

, "get_or_create" SELECT INSERT Postgres.

UNIQUE INSERT ( ) SELECT . , . .

, ( ):

• RETURNING ON CONFLICT PostgreSQL?