Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add empty-valued header #961

Open
curme opened this issue Nov 10, 2020 · 1 comment
Open

add empty-valued header #961

curme opened this issue Nov 10, 2020 · 1 comment

Comments

@curme
Copy link

curme commented Nov 10, 2020

Expected behavior

I want to add a header with empty value, for example, my request requires a header 'Authorization: ' which is a 'Authorization' header valued as ''.

url = 'xxx.xxxx.com'
headers = {'Content-Type': 'application/json', 'Authorization': ''}
self.crawl(url, callback=self.xxx_handle, method='POST', headers=headers)

Actual behavior

However, actually, the header 'Authorization' seems to be kicked out because it's empty.

Maybe, it might cause by lib 'libcurl', which I found linux command 'curl' has the same behavior:
curl --request GET 'https://github.com/binux/pyspider' --header 'Accept: */*' --header 'Authorization: ' -v

In the curl printed log, we could find that 'Authorization' header was removed:
> GET /binux/pyspider HTTP/1.1
> Host: github.com
> User-Agent: curl/7.61.0
> Accept: */*
>

I want to add empty-valued header into request. Any suggestions?
Thanks a lot!

@curme
Copy link
Author

curme commented Nov 11, 2020

I found that curl has already supported sending empty header by replacing the colon with a semicolon. Exactly in the case I provided above, we could do in this way:
curl --request GET 'https://github.com/binux/pyspider' --header 'Accept: */*' --header 'Authorization;' -v

log:
> GET /binux/pyspider HTTP/1.1
> Host: github.com
> User-Agent: curl/7.61.0
> Accept: */*
> Authorization:
>

Which means 'libcurl' enables users to send empty header in request (BTW I suppose pyspider also rely on 'libcurl').
But I still haven found how to do in pyspider. Need your help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant