Is it possible to use the natural key for GenericForeignKey in Django?

Question

I have the following:

target_content_type = models.ForeignKey(ContentType, related_name='target_content_type') target_object_id = models.PositiveIntegerField() target = generic.GenericForeignKey('target_content_type', 'target_object_id') 

I would like dampads - natural ones to emit a natural key for this relationship. Is it possible? If not, is there an alternative strategy that would not bind me to the target primary key?

Answer 1

TL DR . There is currently no sensible way to do this without creating a custom Serializer / Deserializer .

The problem with models that have common relationships is that Django does not see target as a field at all, only target_content_type and target_object_id , and it tries to serialize and deserialize them separately.

The classes responsible for serializing and deserializing Django models are in the django.core.serializers.base and django.core.serializers.python . All the rest ( xml , json and yaml ) apply to any of them (and python continues to base ). Field serialization is performed as follows (irrelevant lines):

  for obj in queryset: for field in concrete_model._meta.local_fields: if field.rel is None: self.handle_field(obj, field) else: self.handle_fk_field(obj, field) 

Here's the first complication: the ContentType foreign key ContentType processed normally, with natural keys, as we expected. But PositiveIntegerField handled by handle_field , which is executed as follows:

 def handle_field(self, obj, field): value = field._get_val_from_obj(obj) # Protected types (ie, primitives like None, numbers, dates, # and Decimals) are passed through as is. All other values are # converted to string first. if is_protected_type(value): self._current[field.name] = value else: self._current[field.name] = field.value_to_string(obj) 

i.e. the only setting option here (subclassing PositiveIntegerField and defining custom value_to_string ) will have no effect since the serializer will not call This. Changing the target_object_id data target_object_id to something larger than an integer is likely to break many other things, so this is not an option.

We could define our custom handle_field to emit natural keys in this case, but then a second complication arises: deserialization is done as follows:

  for (field_name, field_value) in six.iteritems(d["fields"]): field = Model._meta.get_field(field_name) ... data[field.name] = field.to_python(field_value) 

Even if we set up the to_python method, it only field_value , outside the context of the object. This is not a problem when using integers, since it will be interpreted as the primary key of the model , regardless of which model . But for deserializing a natural key, we first need to know which model this key belongs to and that the information is not available if we did not receive a reference to the object (and the target_content_type field target_content_type already been deserialized).

As you can see, this is not an impossible task - supporting natural keys in a generic relationship, but to achieve this, a lot needs to be changed in the serialization and deserialization code. Necessary steps, then (if someone copes with the task):

• Create a custom Field extending PositiveIntegerField using methods for encoding / decoding an object - calling the reference models natural_key and get_by_natural_key ;
• Cancel the handle_field serializer to invoke the encoder, if any;
• Introduce your own deserializer, which: 1) imposes some order in the fields, ensuring that the content type is deserialized in front of the natural key; 2) calls the decoder, passing not only field_value , but also a link to the decoded ContentType .

Answer 2

I wrote a special Serializer and Deserializer that supports GenericFK. Briefly checked it and seems to do the job.

Here is what I came up with:

 import json from django.contrib.contenttypes.generic import GenericForeignKey from django.utils import six from django.core.serializers.json import Serializer as JSONSerializer from django.core.serializers.python import Deserializer as \ PythonDeserializer, _get_model from django.core.serializers.base import DeserializationError import sys class Serializer(JSONSerializer): def get_dump_object(self, obj): dumped_object = super(CustomJSONSerializer, self).get_dump_object(obj) if self.use_natural_keys and hasattr(obj, 'natural_key'): dumped_object['pk'] = obj.natural_key() # Check if there are any generic fk in this obj # and add a natural key to it which will be deserialized by a matching Deserializer. for virtual_field in obj._meta.virtual_fields: if type(virtual_field) == GenericForeignKey: content_object = getattr(obj, virtual_field.name) dumped_object['fields'][virtual_field.name + '_natural_key'] = content_object.natural_key() return dumped_object def Deserializer(stream_or_string, **options): """ Deserialize a stream or string of JSON data. """ if not isinstance(stream_or_string, (bytes, six.string_types)): stream_or_string = stream_or_string.read() if isinstance(stream_or_string, bytes): stream_or_string = stream_or_string.decode('utf-8') try: objects = json.loads(stream_or_string) for obj in objects: Model = _get_model(obj['model']) if isinstance(obj['pk'], (tuple, list)): o = Model.objects.get_by_natural_key(*obj['pk']) obj['pk'] = o.pk # If has generic fk's, find the generic object by natural key, and set it's # pk according to it. for virtual_field in Model._meta.virtual_fields: if type(virtual_field) == GenericForeignKey: natural_key_field_name = virtual_field.name + '_natural_key' if natural_key_field_name in obj['fields']: content_type = getattr(o, virtual_field.ct_field) content_object_by_natural_key = content_type.model_class().\ objects.get_by_natural_key(obj['fields'][natural_key_field_name][0]) obj['fields'][virtual_field.fk_field] = content_object_by_natural_key.pk for obj in PythonDeserializer(objects, **options): yield obj except GeneratorExit: raise except Exception as e: # Map to deserializer error six.reraise(DeserializationError, DeserializationError(e), sys.exc_info()[2])