Allow numpy values in query

[osindm]

Describe the bug

Passing a numpy integer value (numpy.int16, numpy.int32 , numpy.int64 etc.) in dictionary as GET request 'params' gives an TypeError: Invalid variable type: value should be str, int or float, got 12345 of type <class 'numpy.int16'>

import numpy as np
import aiohttp 

number = np.int16(12345) 
params = {"number": number}

async with aiohttp.ClientSession() as session: 
    async with session.get(url, params=params)  as resp: 
        await resp.text()

But it works properly if passing numpy integer values in a dictionary as POST request 'data'.

To Reproduce

Place any numpy integer value (numpy.int16, numpy.int32 , numpy.int64 etc.) in GET request 'params'

Expected behavior

To accept numpy integer type in GET request 'params'

Logs/tracebacks

TypeError: Invalid variable type: value should be str, int or float, got 12345 of type <class 'numpy.int16'>

Python Version

$ python --version
Version: 3.8.8

aiohttp Version

$ python -m pip show aiohttp
Version: 3.8.5

multidict Version

$ python -m pip show multidict
Version: 6.0.4

yarl Version

$ python -m pip show yarl
Version: 1.9.2

OS

Windows

Related component

Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[Dreamsorcerer]

Relevant code:

yarl/yarl/_url.py

Lines 935 to 951 in ea09409

	def _query_var(v):
	cls = type(v)
	if issubclass(cls, str):
	return v
	if issubclass(cls, float):
	if math.isinf(v):
	raise ValueError("float('inf') is not supported")
	if math.isnan(v):
	raise ValueError("float('nan') is not supported")
	return str(float(v))
	if issubclass(cls, int) and cls is not bool:
	return str(int(v))
	raise TypeError(
	"Invalid variable type: value "
	"should be str, int or float, got {!r} "
	"of type {}".format(v, cls)
	)

[webknjaz]

I don't think we should accept types that we don't know how to correctly represent as strings. Same as booleans and other unobvious contexts.
-1 from me.

[Dreamsorcerer]

Yeah, we'd need to import numpy for this to work, which doesn't seem reasonable.
It would make more sense for numpy to make these numbers more compatible. If they were to subclass int, then the code would work just fine. Alternatively, if it were to be a subclass of numbers.Number, we could consider it (although there has been discussion of abandoning the numbers module as it doesn't work with static typing).

In the current state, numpy values are just custom, unique types that mean nothing to yarl.

[osindm]

However numpy.int16, numpy.int32 etc. are subclasses of numbers.Number.
More precisely numpy.intXX are subclasses of numbers.Integral (numbers.Integral is subclass of number.Number).
The numbers.Integral is interest because it has a conversion to int.
I supposed it would be a variant to check:

if issubclass(cls, Integral) and cls is not bool: 
         return str(int(v)) 

Or may be (at the worst) to delete checking for int types completely and try to convert to int forcibly after the all other type checking (and we would catch an exception if custom type can not be convert to int type).

It would be valuable to allow numpy values because numpy is widely used.

[webknjaz]

The thing is that URLs are strings and they only operate on strings. Accepting int is more of a convenience. Also, it's a primitive that's always available. We can't special-case an unlimited number of types. Kinda regret not going with just strings as they are the only ones guaranteed not to be messed up.
What you suggest allows for any Integral subclass with __str__() returning nonsense to come through. Do we want that really? What do other libs do? Is there any comparison?
cc @Dreamsorcerer

[Dreamsorcerer]

However numpy.int16, numpy.int32 etc. are subclasses of numbers.Number.

Oh, weird, when I tested it, it came up False. I've just upgraded to latest version and retested and it is True, maybe something changed in a recent release (or maybe I just tested it incorrectly...).

What do other libs do? Is there any comparison?

Well, urllib.parse.urlencode() seems to accept anything.
As does requests (maybe it uses urllib.parse?).

>>> class Foo:
...   def __str__(self):
...    return "BANANA"
... 
>>> urllib.parse.urlencode({"foo": Foo()})
'foo=BANANA'
>>> r=requests.get("https://google.com", params={"foo": Foo()})
>>> r.url
'https://www.google.com/?foo=BANANA'

Reserved characters are escaped:

>>> class Foo:
...   def __str__(self):
...    return "BANANA&x=y"
... 
>>> urllib.parse.urlencode({"foo": Foo()})                     
'foo=BANANA%26x%3Dy'

Personally, I don't mind either way, as long as it doesn't allow injecting arguments.

[osindm]

What do other libs do? Is there any comparison?

Requests internally uses urllib3 and accepts numpy values (as well as boolean by the way).

Standalone urllib3 also do it:

import urllib3
import numpy as np

http = urllib3.PoolManager()
resp = http.request(
    "GET",
    "https://google.com",
    fields={"Number": np.int16(12345),
            "Bool": True}
)

print(resp.geturl()) 
# Output:  https://www.google.com/?Number=12345&Bool=True

[webknjaz]

Okay, I guess we could make an exception for Integral numbers. bool would be weird, though.