The request failed schema validation

[maimorag]

Describe the bug

I'm currently trying to locate an email with the MessageID (NTLM, version: 2013). Interestingly, when I utilized exchangelib 5.0.3, I encountered no errors and received the intended message. However, upon upgrading to exchangelib version 5.3.0, I encountered an error when attempting to retrieve emails with version 2013 (although it worked fine with 2013 SP1).

2024-05-20T14:00:46.097963229Z error: (EWS v2_instance_1_EWS v2_ews-search-mailbox) ErrorSchemaValidation: The request failed schema validation. The 'Id' attribute is invalid - The value 'workingset' is invalid according to its datatype 'http://schemas.microsoft.com/exchange/services/2006/types:DistinguishedFolderIdNameType' - The Enumeration constraint failed. (line: 2 position: 944)
Full stacktrace:
Traceback (most recent call last):
  File "<string>", line 14866, in sub_main
  File "<string>", line 14132, in search_items_in_mailbox
  File "<string>", line 14089, in get_limited_number_of_messages_from_qs
  File "/usr/local/lib/python3.10/site-packages/exchangelib/queryset.py", line 270, in __iter__
    yield from self._format_items(items=self._query(), return_format=self.return_format)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/queryset.py", line 345, in _item_yielder
    for i in iterable:
  File "/usr/local/lib/python3.10/site-packages/exchangelib/account.py", line 720, in fetch
    yield from self._consume_item_service(
  File "/usr/local/lib/python3.10/site-packages/exchangelib/account.py", line 409, in _consume_item_service
    is_empty, items = peek(items)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/util.py", line 152, in peek
    first = next(iterable)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/folders/collections.py", line 211, in find_items
    yield from FindItem(account=self.account, page_size=page_size).call(
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 216, in _elems_to_objs
    for elem in elems:
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 800, in _paged_call
    pages = self._get_pages(payload_func, kwargs, len(paging_infos))
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 897, in _get_pages
    page_elems = list(self._get_elements(payload=payload))
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 299, in _get_elements
    yield from self._response_generator(payload=payload)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 783, in _response_generator
    response = self._get_response_xml(payload=payload)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 408, in _get_response_xml
    return self._get_soap_messages(body=body, **parse_opts)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 496, in _get_soap_messages
    self._raise_soap_errors(fault=fault)  # Will throw SOAPError or custom EWS error
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 536, in _raise_soap_errors
    raise vars(errors)[code](msg)
exchangelib.errors.ErrorSchemaValidation: The request failed schema validation. The 'Id' attribute is invalid - The value 'workingset' is invalid according to its datatype 'http://schemas.microsoft.com/exchange/services/2006/types:DistinguishedFolderIdNameType' - The Enumeration constraint failed. (line: 2 position: 944)

I saw that a fix was out in v1.1.1 according to #320, but I'm not sure how and if it is related since this fixed was merged in v1.1.1.

Expected behavior
getting the message with 2013 server version.

Additional context
python 3.10

[ecederstrand]

I believe this was fixed in exchangelib 5.4.0. See #1301

Try upgrading and report back your findings here.

[maimorag]

Thanks @ecederstrand.
After updating to 5.4.0 with the same credentials I got (I didn't get it with the previous versions):

ErrorTimeoutExpired: Reraised from ConnectionError(('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')))
Full stacktrace:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/cached_property.py", line 70, in __get__
    return obj_dict[name]
KeyError: 'root'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 564, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/exchangelib/util.py", line 827, in post_ratelimited
    r = session.post(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 637, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 579, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 14485, in sub_main
  File "<string>", line 14444, in test_module
  File "<string>", line 12883, in get_folder_by_path
  File "/usr/local/lib/python3.10/site-packages/cached_property.py", line 74, in __get__
    return obj_dict.setdefault(name, self.func(obj))
  File "/usr/local/lib/python3.10/site-packages/exchangelib/account.py", line 350, in root
    return Root.get_distinguished(account=self)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/folders/roots.py", line 145, in get_distinguished
    return cls._get_distinguished(
  File "/usr/local/lib/python3.10/site-packages/exchangelib/folders/base.py", line 225, in _get_distinguished
    return cls.resolve(account=folder.account, folder=folder)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/folders/base.py", line 529, in resolve
    folders = list(FolderCollection(account=account, folders=[folder]).resolve())
  File "/usr/local/lib/python3.10/site-packages/exchangelib/folders/collections.py", line 335, in resolve
    yield from self.__class__(account=self.account, folders=resolveable_folders).get_folders(
  File "/usr/local/lib/python3.10/site-packages/exchangelib/folders/collections.py", line 403, in get_folders
    yield from GetFolder(account=self.account).call(
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/get_folder.py", line 51, in _elems_to_objs
    for folder, elem in zip(self.folders, elems):
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 278, in _chunked_get_elements
    yield from self._get_elements(payload=payload_func(chunk, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 326, in _get_elements
    raise e
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 299, in _get_elements
    yield from self._response_generator(payload=payload)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 262, in _response_generator
    response = self._get_response_xml(payload=payload)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 395, in _get_response_xml
    r = self._get_response(payload=payload, api_version=api_version)
  File "/usr/local/lib/python3.10/site-packages/exchangelib/services/common.py", line 346, in _get_response
    r, session = post_ratelimited(
  File "/usr/local/lib/python3.10/site-packages/exchangelib/util.py", line 835, in post_ratelimited
    raise ErrorTimeoutExpired(f"Reraised from {e.__class__.__name__}({e})")
exchangelib.errors.ErrorTimeoutExpired: Reraised from ConnectionError(('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')))

[ecederstrand]

That's just your server resetting the connection. Exchange servers will do that once in a while. If you want exchangelib to handle that gracefully, then add a retry policy: https://ecederstrand.github.io/exchangelib/#fault-tolerance

[maimorag]

ok I see, but why it would reset the connection with version 5.4.0 but when I change it back to using the previous version this error doesn't raise? Is that ok?

[ecederstrand]

Connection errors come and go. It's not necessarily correlated with you installing different versions of exchangelib. Unless this is a persistent error and you cannot get any data at all from the server using 5.4.0 then I don't think there's anything to worry about.

[maimorag]

ok thanks, I'll keep monitoring for the night and let you know.
Thanks for the quick response, and maintaining the library!

[maimorag]

Hey @ecederstrand,
sadly this is a persistent error even when trying to access the account folders(without looking for specific email), any suggestions?
it doesn't repreduces with older versions of exchangelib.

[ecederstrand]

How does the error manifest? Did you add a retry policy? Do you have a stack trace? And does the stack trace happen before the retry policy is set to time out?

The best way to find out what a working and a non-working version of exchangelib do differently is to enable debug logging and comparing the XML requests and responses leading up to the exception.

[JudahSchwartz]

we looked into this doesnt seem related to the library code