This question was asked here Foreign keys on Scrapy without an accepted answer, so I'm here to re-ask the question with a more clearly defined minimum setting:
Django model:
class Article(models.Model):
title = models.CharField(max_length=255)
content = models.TextField()
category = models.ForeignKey('categories.Category', null=True, blank=True)
Note that the definition categoryis not relevant here, but it uses ForeignKey. So, in the django shell, this will work:
c = Article(title="foo", content="bar", category_id=2)
c.save()
Subject of study:
class BotsItem(DjangoItem):
django_model = Article
Tubing:
class BotsPipeline(object):
def process_item(self, item, spider):
item['category_id'] = 2
item.save()
return item
With the code above, scrapy complains:
exceptions.KeyError: 'BotsItem does not support field: category_id'
Fair, because it category_iddoes not appear in the django model, from which we get the scrapy element. For the record, if we have a pipeline (suppose we have a category foo):
class BotsPipeline(object):
def process_item(self, item, spider):
item['category'] = 'foo'
item.save()
return item
Now scrapy complains:
exceptions.TypeError: isinstance() arg 2 must be a class, type, or tuple
of classes and types
So what should we do?