I met a very strange problem running Scrapy with PyCharm:
Debug works with the same configuration, but Run with PyCharm does NOT work.
- Windows 10
- PyCharm 2016.3.3
- Scrapy 1.3.3
- Python 3.6.0
Configuration:
PyCharm Change Configuration Page
When I try to debug scrapy, it works fine. When I try to start scrapy, I get below error:
C:\Users\baib2\AppData\Local\Programs\Python\Python36\python.exe
C:/Users/baib2/AppData/Local/Programs/Python/Python36/Lib/site-packages/scrapy/cmdline.py crawl scenelist_spider
Traceback (most recent call last):
File "C:/Users/baib2/AppData/Local/Programs/Python/Python36/Lib/site-packages/scrapy/cmdline.py", line 8, in <module
import scrapy
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\__init__.py", line 27, in <module
from . import _monkeypatches
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\_monkeypatches.py", line 20, in <module
import twisted.persisted.styles # NOQA
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\twisted\persisted\styles.py", line 21, in <module
from twisted.python.compat import _PY3, _PYPY
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\twisted\python\__init__.py", line 11, in <module
from .compat import unicode
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\twisted\python\compat.py", line 612, in <module
from http import cookiejar as cookielib
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\Lib\site-packages\scrapy\http\__init__.py", line 8, in <module
from scrapy.http.headers import Headers
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\http\__init__.py", line 10, in <module
from scrapy.http.request import Request
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\http\request\__init__.py", line 8, in <module
from w3lib.url import safe_url_string
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\w3lib\url.py", line 17, in <module
from six.moves.urllib.request import pathname2url, url2pathname
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\six.py", line 92, in __get__
result = self._resolve()
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\six.py", line 160, in _resolve
module = _import_module(self.mod)
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\site-packages\six.py", line 82, in _import_module
__import__(name)
File "C:\Users\baib2\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 88, in <module
import http.client
ModuleNotFoundError: No module named 'http.client'
Process finished with exit code 1
There is no need to attach any code, because even execute the command:
python3.exe C:\Users\baib2\AppData\Local\Programs\Python\Python36\Lib\site-packages\scrapy\cmdline.py
will result in the same error.
I checked my sys.path , compared to run, debug has another way:
'C: \ Program Files (x86) \ JetBrains \ PyCharm 2016.3.3 \ helpers \ pydev'
And I really don't think that will make a difference.
, - , !