Foreign key djangoitem screening

This question was asked here Foreign keys on Scrapy without an accepted answer, so I'm here to re-ask the question with a more clearly defined minimum setting:

Django model:

class Article(models.Model):
    title = models.CharField(max_length=255)
    content = models.TextField()
    category = models.ForeignKey('categories.Category', null=True, blank=True)

Note that the definition categoryis not relevant here, but it uses ForeignKey. So, in the django shell, this will work:

c = Article(title="foo", content="bar", category_id=2)
c.save()

Subject of study:

class BotsItem(DjangoItem):
    django_model = Article

Tubing:

class BotsPipeline(object):
    def process_item(self, item, spider):
        item['category_id'] = 2
        item.save()
        return item

With the code above, scrapy complains:

exceptions.KeyError: 'BotsItem does not support field: category_id'

Fair, because it category_iddoes not appear in the django model, from which we get the scrapy element. For the record, if we have a pipeline (suppose we have a category foo):

class BotsPipeline(object):
    def process_item(self, item, spider):
        item['category'] = 'foo'
        item.save()
        return item

Now scrapy complains:

exceptions.TypeError: isinstance() arg 2 must be a class, type, or tuple
 of classes and types

So what should we do?

+4
1

, , . exceptions.TypeError, item['category'] expects an instance of class, in my case I am using [ django-categories ][1] so in the pipeline just replace with this (assume Category` ORM):

class BotsPipeline(object):
    def process_item(self, item, spider):
        item['category'] = Category.objects.get(id=2)
        item.save()
        return item
+7

Source: https://habr.com/ru/post/1545454/


All Articles