Question with Scrapy Newbie - cannot work with training file

Question

I'm a complete newbie to Python and Scrapy, so I started by trying to replicate the tutorial. I am trying to clean the www.dmoz.org website according to the manual.

I will compile dmoz_spider.py as below

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector

from dmoz.items import DmozItem

class DmozSpider(BaseSpider):
   name = "dmoz.org"
   allowed_domains = ["dmoz.org"]
   start_urls = [
       "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
       "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
   ]

   def parse(self, response):
       hxs = HtmlXPathSelector(response)
       sites = hxs.select('//ul/li')
       items = []
       for site in sites:
           item = DmozItem()
           item['title'] = site.select('a/text()').extract()
           item['link'] = site.select('a/@href').extract()
           item['desc'] = site.select('text()').extract()
           items.append(item)
       return items

and what I have to get through the site is something else.
any idea what i mess up?

Answer 1

I had this problem. Make sure you make the next change, as the tutorial says.

Open items.py and see if you change the class

class TutorialItem(Item):
    title=Field()
    link=Field()
    desc=Field()

at

class DmozItem(Item):
    title=Field()
    link=Field()
    desc=Field()

Answer 2

, , . , , ? ( , ...)

Answer 3

You need to go to the directory containing the settings.py file and run

scrapy crawl dmoz from there.

Clear your project structure from https://github.com/scrapy/dirbot for clarity