Scrapy 1.0+ correct access to settings in CsvItemExporter sublcass?

Question

Previously, access to the settings in a subclass of CsvItemExporter , which I need to change the separator specified in the settings, can be done with:

 from scrapy.conf import settings 

However, this method is now depreciating:

ScrapyDeprecationWarning: The scrapy.conf module scrapy.conf deprecated, use the crawler.settings attribute from the scrapy.conf import settings instead.

How can i do this now? The usual methods of the from_crawler , " from_settings do not work in CsvItemExporter .

Answer 1

Assuming everything is set up correctly, unfamiliar with this Trace error, whenever I use CSVitemexporter, I do this by creating an additional module with which the project is CSVitemexporter, and then just just specify my separator ..

yournameformodule.py

 from scrapy.conf import settings from scrapy.contrib.exporter import CsvItemExporter class MyProjectCsvItemExporter(CsvItemExporter): def __init__(self, *args, **kwargs): delimiter = settings.get('CSV_DELIMITER', ',') kwargs['delimiter'] = delimiter fields_to_export = settings.get('FIELDS_TO_EXPORT', []) if fields_to_export : kwargs['fields_to_export'] = fields_to_export super(MyProjectCsvItemExporter, self).__init__(*args, **kwargs) 

then make sure that you specify items in your settings (also in your items.py)

settings.py

 FEED_EXPORTERS = { 'csv': 'PROJECTNAME.YOURNAMEFORMODULE.MyProjectCsvItemExporter', } FIELDS_TO_EXPORT = [ 'etc', 'etc2',] 

The only thing I’m sure is that the process works in the same way or as a spider, as if it were a scanning spider, although I do not understand why not, I have not tested yet, except for using the finder. If you are still stuck on these life keys with your projects, to better help you.

Answer 2

If there is no other way to access the settings, you can try replacing

 from scrapy.conf import settings 

with

 from scrapy.utils.project import get_project_settings settings = get_project_settings() 

Essentially, this is what the current compatibility in scrapy/conf.py (It still exists at this point, 3 years later :)