We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crawl_config = { 'proxy': 't13177827711279:24qefecw@tps158.kdlapi.com:15818', #'proxy':'t13038971957781:fqtmsm8i@tps187.kdlapi.com:15818' } 我配置配置了快代理的隧道代理。在调试时代理能够正常工作。 但是在爬虫run起来之后,代理却没有生效。 我判断代理没有生效的原因是: 1.快代理网站对我购买的隧道代理的监控中,代理使用次数没有增加 图中零星的代理使用,是在调试过程,尝试爬取页面时的代理正确生效。 2.在Active Tasks任务的详情内容中,fatch中,无代理的配置。 fetch { "save": { "key": "肯尼亚航空" } }
crawl_config = { 'proxy': 't13177827711279:24qefecw@tps158.kdlapi.com:15818', #'proxy':'t13038971957781:fqtmsm8i@tps187.kdlapi.com:15818' }
Active Tasks
fatch
fetch { "save": { "key": "肯尼亚航空" } }
我尝试使用下述配置启动,同样无效
pyspider --phantomjs-proxy 用户名:密码@tps158.kdlapi.com:15818
The text was updated successfully, but these errors were encountered:
No branches or pull requests
crawl_config = { 'proxy': 't13177827711279:24qefecw@tps158.kdlapi.com:15818', #'proxy':'t13038971957781:fqtmsm8i@tps187.kdlapi.com:15818' }
我配置配置了快代理的隧道代理。在调试时代理能够正常工作。
但是在爬虫run起来之后,代理却没有生效。
我判断代理没有生效的原因是:
1.快代理网站对我购买的隧道代理的监控中,代理使用次数没有增加
图中零星的代理使用,是在调试过程,尝试爬取页面时的代理正确生效。
2.在
Active Tasks
任务的详情内容中,fatch
中,无代理的配置。fetch { "save": { "key": "肯尼亚航空" } }
我尝试使用下述配置启动,同样无效
The text was updated successfully, but these errors were encountered: