You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In my scrapy app, I use scrapy -selenium with geckodriver. When I try use command 'scrapy-deploy', I get an error:
file "/tmp/scrapy_job_it-1590437526-tezy3xue.egg/scrapy_job_it/spiders/bdcrawler.py", line 3, in \nModuleNotFoundError: No module named 'scrapy_selenium'\n"}
What can I do to deploy app with selenium?
All track for error is here:
Server response (200):
{"node_name": "cbb42d40-da02-4d65-bb04-ff436c5aacc1", "status": "error", "message": "/app/.heroku/python/lib/python3.6/site-packages/scrapy/utils/project.py:94: ScrapyDeprecationWarning: Use of environment variables prefixed with SCRAPY_ to override settings is deprecated. The following environment variables are currently defined: EGG_VERSION\n ScrapyDeprecationWarning\nTraceback (most recent call last):\n File "/app/.heroku/python/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n "main", mod_spec)\n File "/app/.heroku/python/lib/python3.6/runpy.py", line 85, in _run_code\n exec(code, run_globals)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapyd/runner.py", line 40, in \n main()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapyd/runner.py", line 37, in main\n execute()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/cmdline.py", line 142, in execute\n cmd.crawler_process = CrawlerProcess(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 280, in init\n super(CrawlerProcess, self).init(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 152, in init\n self.spider_loader = self._get_spider_loader(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 146, in _get_spider_loader\n return loader_cls.from_settings(settings.frozencopy())\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 60, in from_settings\n return cls(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 24, in init\n self._load_all_spiders()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 46, in _load_all_spiders\n for module in walk_modules(name):\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/utils/misc.py", line 77, in walk_modules\n submod = import_module(fullpath)\n File "/app/.heroku/python/lib/python3.6/importlib/init.py", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File "", line 994, in _gcd_import\n File "", line 971, in _find_and_load\n File "", line 955, in _find_and_load_unlocked\n File "", line 656, in _load_unlocked\n File "", line 626, in _load_backward_compatible\n File "/tmp/scrapy_job_it-1590437526-tezy3xue.egg/scrapy_job_it/spiders/bdcrawler.py", line 3, in \nModuleNotFoundError: No module named 'scrapy_selenium'\n"}
The text was updated successfully, but these errors were encountered:
In my scrapy app, I use scrapy -selenium with geckodriver. When I try use command 'scrapy-deploy', I get an error:
file "/tmp/scrapy_job_it-1590437526-tezy3xue.egg/scrapy_job_it/spiders/bdcrawler.py", line 3, in \nModuleNotFoundError: No module named 'scrapy_selenium'\n"}
What can I do to deploy app with selenium?
All track for error is here:
Server response (200):
{"node_name": "cbb42d40-da02-4d65-bb04-ff436c5aacc1", "status": "error", "message": "/app/.heroku/python/lib/python3.6/site-packages/scrapy/utils/project.py:94: ScrapyDeprecationWarning: Use of environment variables prefixed with SCRAPY_ to override settings is deprecated. The following environment variables are currently defined: EGG_VERSION\n ScrapyDeprecationWarning\nTraceback (most recent call last):\n File "/app/.heroku/python/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n "main", mod_spec)\n File "/app/.heroku/python/lib/python3.6/runpy.py", line 85, in _run_code\n exec(code, run_globals)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapyd/runner.py", line 40, in \n main()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapyd/runner.py", line 37, in main\n execute()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/cmdline.py", line 142, in execute\n cmd.crawler_process = CrawlerProcess(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 280, in init\n super(CrawlerProcess, self).init(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 152, in init\n self.spider_loader = self._get_spider_loader(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 146, in _get_spider_loader\n return loader_cls.from_settings(settings.frozencopy())\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 60, in from_settings\n return cls(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 24, in init\n self._load_all_spiders()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 46, in _load_all_spiders\n for module in walk_modules(name):\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/utils/misc.py", line 77, in walk_modules\n submod = import_module(fullpath)\n File "/app/.heroku/python/lib/python3.6/importlib/init.py", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File "", line 994, in _gcd_import\n File "", line 971, in _find_and_load\n File "", line 955, in _find_and_load_unlocked\n File "", line 656, in _load_unlocked\n File "", line 626, in _load_backward_compatible\n File "/tmp/scrapy_job_it-1590437526-tezy3xue.egg/scrapy_job_it/spiders/bdcrawler.py", line 3, in \nModuleNotFoundError: No module named 'scrapy_selenium'\n"}
The text was updated successfully, but these errors were encountered: