Investigate AIT errors for `PUT /agents/upgrade_custom` test #23463

fdalmaup · 2024-05-16T09:28:02Z

Description

In #23447 (review) it was found that the test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom test is failing with the following error:

Error

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.3.1, pluggy-1.2.0 -- /usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.10.6', 'Platform': 'Linux-5.19.0-1028-aws-x86_64-with-glibc2.35', 'Packages': {'pytest': '7.3.1', 'pluggy': '1.2.0'}, 'Plugins': {'asyncio': '0.18.1', 'html': '2.1.1', 'trio': '0.7.0', 'metadata': '3.0.0', 'tavern': '1.23.5', 'aiohttp': '1.0.4'}}
rootdir: /tmp/Test_integration_endpoints_B4551_test_files/api/test/integration
configfile: pytest.ini
plugins: asyncio-0.18.1, html-2.1.1, trio-0.7.0, metadata-3.0.0, tavern-1.23.5, aiohttp-1.0.4
asyncio: mode=auto
collecting ... collected 10 items

test_agent_PUT_endpoints.tavern.yaml::PUT /agents/group PASSED           [ 10%]
test_agent_PUT_endpoints.tavern.yaml::PUT /groups/{group_id}/configuration PASSED [ 20%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/group/{group_id}/restart PASSED [ 30%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/reconnect PASSED       [ 40%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/restart PASSED         [ 50%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/group/{group_id} PASSED [ 60%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/restart PASSED [ 70%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/node/{node_id}/restart PASSED [ 80%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/upgrade PASSED         [ 90%]
test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom FAILED [100%]

=================================== FAILURES ===================================
_ /tmp/Test_integration_endpoints_B4551_test_files/api/test/integration/test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom _
Format variables:
  protocol:s = 'https'
  host:s = 'localhost'
  port:d = '55000'
  test_login_token = 'eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ3YXp1aCIsImF1ZCI6IldhenVoIEFQSSBSRVNUIiwibmJmIjoxNzE1NzkxNTg4LCJleHAiOjE3MTU3OTI0ODgsInN1YiI6InRlc3RpbmciLCJydW5fYXMiOmZhbHNlLCJyYmFjX3JvbGVzIjpbMV0sInJiYWNfbW9kZSI6IndoaXRlIn0.AXilsqZT9BZxMJXEaJUhB0kn3qERg8_wt3oY9ANTxQKhayDXvWMxspskl0RJcZKZXy33ZCAN04y6dww5EZ6k2eDkARURsWBEFSiXPeyARQIOsxTsvklXVTDI4GnkMGNOLVBCHgzJS_tUu-MAcqjpNHvCDyad3BzrdrIMeAF7tMmpo4dX'
  custom_wpk_path:s = '/var/ossec/test.wpk'

Source test stage (line 1237):
  - name: Try to customly upgrade an agent selecting installer (WPK does not exist)
    request:
      verify: False
      url: "{protocol:s}://{host:s}:{port:d}/agents/upgrade_custom"
      method: PUT
      headers:
        Authorization: "Bearer {test_login_token}"
      params:
        agents_list: '005'
        installer: upgrade.sh
        file_path: "{custom_wpk_path:s}"
    response:
      status_code: 200
      json:
        data:
          affected_items:
            - agent: '005'
              task_id: !anyint
          total_affected_items: 1
          total_failed_items: 0
          failed_items: [ ]
        message: !anystr


Formatted stage:
  name: Try to customly upgrade an agent selecting installer (WPK does not exist)
  request:
    headers:
      Authorization: Bearer eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ3YXp1aCIsImF1ZCI6IldhenVoIEFQSSBSRVNUIiwibmJmIjoxNzE1NzkxNTg4LCJleHAiOjE3MTU3OTI0ODgsInN1YiI6InRlc3RpbmciLCJydW5fYXMiOmZhbHNlLCJyYmFjX3JvbGVzIjpbMV0sInJiYWNfbW9kZSI6IndoaXRlIn0.AXilsqZT9BZxMJXEaJUhB0kn3qERg8_wt3oY9ANTxQKhayDXvWMxspskl0RJcZKZXy33ZCAN04y6dww5EZ6k2eDkARURsWBEFSiXPeyARQIOsxTsvklXVTDI4GnkMGNOLVBCHgzJS_tUu-MAcqjpNHvCDyad3BzrdrIMeAF7tMmpo4dX
    method: PUT
    params:
      agents_list: '005'
      file_path: '/var/ossec/test.wpk'
      installer: upgrade.sh
    url: 'https://localhost:55000/agents/upgrade_custom'
    verify: false
  response:
    json:
      data:
        affected_items:
        - agent: '005'
          task_id: !anyint ''
        failed_items: []
        total_affected_items: 1
        total_failed_items: 0
      message: !anystr ''
    status_code: 200

Errors:
E   tavern.util.exceptions.TestFailError: Test 'Try to customly upgrade an agent selecting installer (WPK does not exist)' failed:
    - List item(s) not present in response: [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}]
------------------------------ Captured log call -------------------------------
ERROR    tavern.response.base:base.py:41 List item(s) not present in response: [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}]
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/util/dict_util.py", line 406, in check_keys_match_recursive
    assert actual_val == expected_val
AssertionError: assert {'data': {'affected_items': [], 'total_affected_items': 0, 'total_failed_items': 1, 'failed_items': [{'error': {'code': 1820, 'message': 'Upgrade procedure could not start. Agent already upgrading'}, 'id': ['005']}]}, 'message': 'No upgrade task was created', 'error': 1} == {'data': {'affected_items': [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}], 'total_affected_items': 1, 'total_failed_items': 0, 'failed_items': []}, 'message': <tavern.util.loader.StrSentinel object at 0x7f1a28952920>}
  Differing items:
  {'message': 'No upgrade task was created'} != {'message': <tavern.util.loader.StrSentinel object at 0x7f1a28952920>}
  {'data': {'affected_items': [], 'failed_items': [{'error': {'code': 1820, 'message': 'Upgrade procedure could not start. Agent already upgrading'}, 'id': ['005']}], 'total_affected_items': 0, 'total_failed_items': 1}} != {'data': {'affected_items': [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}], 'failed_items': [], 'total_affected_items': 1, 'total_failed_items': 0}}
  Left contains 1 more item:
  {'error': 1}
  Full diff:
    {
  -  'data': {'affected_items': [{'agent': '005',
  ?                              ^^^^^^^^^^^^^^^
  +  'data': {'affected_items': [],
  ?                              ^
  -                               'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}],
  -           'failed_items': [],
  +           'failed_items': [{'error': {'code': 1820,
  +                                       'message': 'Upgrade procedure could not '
  +                                                  'start. Agent already '
  +                                                  'upgrading'},
  +                             'id': ['005']}],
  -           'total_affected_items': 1,
  ?                                   ^
  +           'total_affected_items': 0,
  ?                                   ^
  -           'total_failed_items': 0},
  ?                                 ^
  +           'total_failed_items': 1},
  ?                                 ^
  -  'message': <tavern.util.loader.StrSentinel object at 0x7f1a28952920>,
  +  'error': 1,
  +  'message': 'No upgrade task was created',
    }

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/util/dict_util.py", line 406, in check_keys_match_recursive
    assert actual_val == expected_val
AssertionError: assert {'affected_items': [], 'total_affected_items': 0, 'total_failed_items': 1, 'failed_items': [{'error': {'code': 1820, 'message': 'Upgrade procedure could not start. Agent already upgrading'}, 'id': ['005']}]} == {'affected_items': [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}], 'total_affected_items': 1, 'total_failed_items': 0, 'failed_items': []}
  Differing items:
  {'total_failed_items': 1} != {'total_failed_items': 0}
  {'failed_items': [{'error': {'code': 1820, 'message': 'Upgrade procedure could not start. Agent already upgrading'}, 'id': ['005']}]} != {'failed_items': []}
  {'affected_items': []} != {'affected_items': [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}]}
  {'total_affected_items': 0} != {'total_affected_items': 1}
  Full diff:
    {
  -  'affected_items': [{'agent': '005',
  -                      'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}],
  -  'failed_items': [],
  ?    ^^^
  +  'affected_items': [],
  ?   + ^^^^
  +  'failed_items': [{'error': {'code': 1820,
  +                              'message': 'Upgrade procedure could not start. '
  +                                         'Agent already upgrading'},
  +                    'id': ['005']}],
  -  'total_affected_items': 1,
  ?                          ^
  +  'total_affected_items': 0,
  ?                          ^
  -  'total_failed_items': 0,
  ?                        ^
  +  'total_failed_items': 1,
  ?                        ^
    }

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/util/dict_util.py", line 406, in check_keys_match_recursive
    assert actual_val == expected_val
AssertionError: assert [] == [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}]
  Right contains one more item: {'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}
  Full diff:
    [
  +  ,
  -  {'agent': '005',
  -   'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>},
    ]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/response/base.py", line 97, in recurse_check_key_match
    check_keys_match_recursive(expected_block, block, [], strict)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/util/dict_util.py", line 464, in check_keys_match_recursive
    check_keys_match_recursive(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/util/dict_util.py", line 464, in check_keys_match_recursive
    check_keys_match_recursive(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/tavern/util/dict_util.py", line 516, in check_keys_match_recursive
    raise exceptions.KeyMismatchError(msg) from e
tavern.util.exceptions.KeyMismatchError: List item(s) not present in response: [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f1a289489d0>}]
=============================== warnings summary ===============================
../../../../../home/ubuntu/.local/lib/python3.10/site-packages/_pytest/nodes.py:642
  /home/ubuntu/.local/lib/python3.10/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

test_agent_PUT_endpoints.tavern.yaml: 10 warnings
  <frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead

test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom
  /home/ubuntu/.local/lib/python3.10/site-packages/pytest_trio/plugin.py:62: TrioDeprecationWarning: trio.MultiError is deprecated since Trio 0.22.0; use BaseExceptionGroup (on Python 3.11 and later) or exceptiongroup.BaseExceptionGroup (earlier versions) instead (https://github.com/python-trio/trio/issues/2211)
    if issubclass(call.excinfo.type, trio.MultiError):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated html file: file:///tmp/Test_integration_endpoints_B4551_test_files/api/test/integration/Test_integration_endpoints_B4551_test_agent_PUT_endpoints.html -
=========================== short test summary info ============================
FAILED test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom
============= 1 failed, 9 passed, 12 warnings in 833.71s (0:13:53) =============

The changes introduced in said PR are not related to the endpoint and should not affect it, so the test was also run for v4.7.4 resulting in the same failure.
The origin may be related to the PUT /agents/upgrade test case, which was successful and involved the same agent and may have generated the unexpected behavior in the upgrade task that triggered the subsequent error.

Checks

The following elements have been updated or reviewed (should also be checked if no modification is required):

Tests (unit tests, API integration tests).
Changelog.
Documentation.
Integration test mapping (using api/test/integration/mapping/_test_mapping.py).

The text was updated successfully, but these errors were encountered:

fdalmaup · 2024-05-17T15:46:40Z

Issue Update

Relaunched the tests locally several times with the 4.7.5 branch and are passing without failures:

$ pytest test_agent_PUT_endpoints.tavern.yaml 
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-7.3.1, pluggy-0.13.1
rootdir: /home/fdalmau/git/wazuh/api/test/integration
configfile: pytest.ini
plugins: aiohttp-1.0.4, trio-0.7.0, metadata-2.0.2, tavern-1.23.5, asyncio-0.18.1, html-2.1.1
asyncio: mode=auto
collected 10 items                                                                                                                                                                                        

test_agent_PUT_endpoints.tavern.yaml ..........                                                                                                                                                     [100%]

============================================================================================ warnings summary =============================================================================================
../../../../../venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================================================== 10 passed, 1 warning in 1270.56s (0:21:10) ================================================================================

To double check, I relaunched the tests for the 4.7.4 version finding the failure:

root@wazuh-master:/# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.7.4"
WAZUH_REVISION="40717"
WAZUH_TYPE="server"

Failed test execution

$ pytest test_agent_PUT_endpoints.tavern.yaml
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-7.3.1, pluggy-0.13.1
rootdir: /home/fdalmau/git/wazuh/api/test/integration
configfile: pytest.ini
plugins: aiohttp-1.0.4, trio-0.7.0, metadata-2.0.2, tavern-1.23.5, asyncio-0.18.1, html-2.1.1
asyncio: mode=auto
collected 10 items                                                                                                                                                                                        

test_agent_PUT_endpoints.tavern.yaml .........F                                                                                                                     [100%]

================================================================================ FAILURES =================================================================================
________________________ /home/fdalmau/git/wazuh/api/test/integration/test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom _________________________
Format variables:
  protocol:s = 'https'
  host:s = 'localhost'
  port:d = '55000'
  test_login_token = 'eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ3YXp1aCIsImF1ZCI6IldhenVoIEFQSSBSRVNUIiwibmJmIjoxNzE1OTUwNTU0LCJleHAiOjE3MTU5NTE0NTQsInN1YiI6InRlc3RpbmciLCJydW5fYXMiOmZhbHNlLCJyYmFjX3JvbGVzIjpbMV0sInJiYWNfbW9kZSI6IndoaXRlIn0.AfELA2avwzzYLa39G0VabsYH15zqtbrBKx8WHonsAuFwOT5ucDiSiAbshwomGuztMuXsv5Xjfm3hwX0nkg8p_t3SAJmOeU_6GmvWeteWNgxxRH1hS8mHGGmCrOdLdDtZeTjEG98KJX_uNG84I-z5dj_S3hYiVsZXVzVGjbwDUtwkfeXl'
  custom_wpk_path:s = '/var/ossec/test.wpk'

Source test stage (line 1237):
  - name: Try to customly upgrade an agent selecting installer (WPK does not exist)
    request:
      verify: False
      url: "{protocol:s}://{host:s}:{port:d}/agents/upgrade_custom"
      method: PUT
      headers:
        Authorization: "Bearer {test_login_token}"
      params:
        agents_list: '005'
        installer: upgrade.sh
        file_path: "{custom_wpk_path:s}"
    response:
      status_code: 200
      json:
        data:
          affected_items:
            - agent: '005'
              task_id: !anyint
          total_affected_items: 1
          total_failed_items: 0
          failed_items: [ ]
        message: !anystr


Formatted stage:
  name: Try to customly upgrade an agent selecting installer (WPK does not exist)
  request:
    headers:
      Authorization: Bearer eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ3YXp1aCIsImF1ZCI6IldhenVoIEFQSSBSRVNUIiwibmJmIjoxNzE1OTUwNTU0LCJleHAiOjE3MTU5NTE0NTQsInN1YiI6InRlc3RpbmciLCJydW5fYXMiOmZhbHNlLCJyYmFjX3JvbGVzIjpbMV0sInJiYWNfbW9kZSI6IndoaXRlIn0.AfELA2avwzzYLa39G0VabsYH15zqtbrBKx8WHonsAuFwOT5ucDiSiAbshwomGuztMuXsv5Xjfm3hwX0nkg8p_t3SAJmOeU_6GmvWeteWNgxxRH1hS8mHGGmCrOdLdDtZeTjEG98KJX_uNG84I-z5dj_S3hYiVsZXVzVGjbwDUtwkfeXl
    method: PUT
    params:
      agents_list: '005'
      file_path: '/var/ossec/test.wpk'
      installer: upgrade.sh
    url: 'https://localhost:55000/agents/upgrade_custom'
    verify: false
  response:
    json:
      data:
        affected_items:
        - agent: '005'
          task_id: !anyint ''
        failed_items: []
        total_affected_items: 1
        total_failed_items: 0
      message: !anystr ''
    status_code: 200

Errors:
E   tavern.util.exceptions.TestFailError: Test 'Try to customly upgrade an agent selecting installer (WPK does not exist)' failed:
    - List item(s) not present in response: [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f65fb5909d0>}]
---------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------
ERROR    tavern.response.base:base.py:41 List item(s) not present in response: [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f65fb5909d0>}]
Traceback (most recent call last):
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/util/dict_util.py", line 406, in check_keys_match_recursive
    assert actual_val == expected_val
AssertionError: assert {'data': {'af... was created'} == {'data': {'af...7f65fb590e20>}
  Differing items:
  {'message': 'No upgrade task was created'} != {'message': <tavern.util.loader.StrSentinel object at 0x7f65fb590e20>}
  {'data': {'affected_items': [], 'failed_items': [{'error': {'code': 1820, 'message': 'Upgrade procedure could not start. Agent already upgrading'}, 'id': ['005']}], 'total_affected_items': 0, 'total_failed_items': 1}} != {'data': {'affected_items': [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f65fb5909d0>}], 'failed_items': [], 'total_affected_items': 1, 'total_failed_items': 0}}
  Left contains 1 more item:
  {'error': 1}
  Use -v to get more diff

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/util/dict_util.py", line 406, in check_keys_match_recursive
    assert actual_val == expected_val
AssertionError: assert {'affected_it...led_items': 1} == {'affected_it...led_items': 0}
  Differing items:
  {'affected_items': []} != {'affected_items': [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f65fb5909d0>}]}
  {'failed_items': [{'error': {'code': 1820, 'message': 'Upgrade procedure could not start. Agent already upgrading'}, 'id': ['005']}]} != {'failed_items': []}
  {'total_failed_items': 1} != {'total_failed_items': 0}
  {'total_affected_items': 0} != {'total_affected_items': 1}
  Use -v to get more diff

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/util/dict_util.py", line 406, in check_keys_match_recursive
    assert actual_val == expected_val
AssertionError: assert [] == [{'agent': '0...f65fb5909d0>}]
  Right contains one more item: {'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f65fb5909d0>}
  Use -v to get more diff

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/response/base.py", line 97, in recurse_check_key_match
    check_keys_match_recursive(expected_block, block, [], strict)
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/util/dict_util.py", line 464, in check_keys_match_recursive
    check_keys_match_recursive(
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/util/dict_util.py", line 464, in check_keys_match_recursive
    check_keys_match_recursive(
  File "/home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/tavern/util/dict_util.py", line 516, in check_keys_match_recursive
    raise exceptions.KeyMismatchError(msg) from e
tavern.util.exceptions.KeyMismatchError: List item(s) not present in response: [{'agent': '005', 'task_id': <tavern.util.loader.IntSentinel object at 0x7f65fb5909d0>}]
============================================================================ warnings summary =============================================================================
../../../../../venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/pytest_trio/plugin.py:62: TrioDeprecationWarning: trio.MultiError is deprecated since Trio 0.22.0; use BaseExceptionGroup (on Python 3.11 and later) or exceptiongroup.BaseExceptionGroup (earlier versions) instead (https://github.com/python-trio/trio/issues/2211)
    if issubclass(call.excinfo.type, trio.MultiError):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info =========================================================================
FAILED test_agent_PUT_endpoints.tavern.yaml::PUT /agents/{agent_id}/upgrade_custom
========================================================== 1 failed, 9 passed, 2 warnings in 1144.35s (0:19:04) ===========================================================

The 1820 error code comes from adding the base 1810 error code with the value provided by the upgrade task corresponding to WM_UPGRADE_UPGRADE_ALREADY_IN_PROGRESS.

wazuh/src/wazuh_modules/agent_upgrade/manager/wm_agent_upgrade_manager.h

Lines 28 to 39 in f094086

    
           typedef enum _wm_upgrade_error_code { 
        
               WM_UPGRADE_SUCCESS = 0, 
        
               WM_UPGRADE_PARSING_ERROR, 
        
               WM_UPGRADE_PARSING_REQUIRED_PARAMETER, 
        
               WM_UPGRADE_TASK_CONFIGURATIONS, 
        
               WM_UPGRADE_TASK_MANAGER_COMMUNICATION, 
        
               WM_UPGRADE_TASK_MANAGER_FAILURE, 
        
               WM_UPGRADE_GLOBAL_DB_FAILURE, 
        
               WM_UPGRADE_INVALID_ACTION_FOR_MANAGER, 
        
               WM_UPGRADE_AGENT_IS_NOT_ACTIVE, 
        
               WM_UPGRADE_SYSTEM_NOT_SUPPORTED, 
        
               WM_UPGRADE_UPGRADE_ALREADY_IN_PROGRESS,

Left the environment up and checked that the status of the task for the agent 005 was Updating (with the GET /agents/upgrade_result endpoint):

 {
        "message": "Success",
        "agent": "005",
        "task_id": 1,
        "node": "master-node",
        "module": "upgrade_module",
        "command": "upgrade",
        "status": "Updating",
        "create_time": "2024-05-17T12:55:53Z",
        "update_time": "2024-05-17T12:55:53Z"
}

And after less than a minute, the task status was Error:

{
        "message": "Success",
        "agent": "005",
        "task_id": 1,
        "node": "master-node",
        "module": "upgrade_module",
        "command": "upgrade",
        "status": "Error",
        "error_msg": "Send upgrade command error",
        "create_time": "2024-05-17T12:55:53Z",
        "update_time": "2024-05-17T12:56:40Z"
}

The master node showed the following logs regarding the task:

2024/05/17 12:45:59 wazuh-modulesd:agent-upgrade: INFO: (8153): Module Agent Upgrade started.
2024/05/17 12:55:53 wazuh-modulesd:agent-upgrade: WARNING: (8160): There are no valid agents to upgrade.
2024/05/17 12:55:55 wazuh-modulesd:agent-upgrade: WARNING: (8160): There are no valid agents to upgrade.
2024/05/17 12:56:40 wazuh-modulesd:agent-upgrade: ERROR: (8116): Error response from agent: 'Could not verify signature'

Conclusion

No changes are directly related to this behavior between 4.7.4 and 4.7.5. We must keep in mind that this error can occur and that it is an internal behavior of the upgrade task that takes longer than expected and generates the error in the test.

GGP1 · 2024-05-17T18:31:56Z

Review

The investigation was carried out on versions 4.7.4 and 4.7.5, resulting in the reproduction of the error only in the first one. The error appears to be caused by delays in the agent's upgrade tasks that does not complete before the test executes the request.

I think we should try to avoid future failures by using the already created (but currently unused) upgrade_delay variable from the AIT environment to give the task a little bit more time to finish. It would be in the previous upgrade performed on that agent, similar to what we do with reconnect_delay.

fdalmaup · 2024-05-20T09:31:45Z

Issue Update

`v4.7.4`

Applied the suggested test configuration in a temporary branch derived from v4.7.4 and the tests passed:

(integrationtest-env) fdalmau@wazuhFW:~/.../api/test/integration(temp-23463-upgrade-agent-ait)$ pytest test_agent_PUT_endpoints.tavern.yaml --nobuild
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-7.3.1, pluggy-0.13.1
rootdir: /home/fdalmau/git/wazuh/api/test/integration
configfile: pytest.ini
plugins: aiohttp-1.0.4, trio-0.7.0, metadata-2.0.2, tavern-1.23.5, asyncio-0.18.1, html-2.1.1
asyncio: mode=auto
collected 10 items                                                                                                                                                                                        

test_agent_PUT_endpoints.tavern.yaml ..........                                                                                                                                                     [100%]

============================================================================================ warnings summary =============================================================================================
../../../../../venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================ 10 passed, 1 warning in 699.68s (0:11:39) ================================================================================
(integrationtest-env) fdalmau@wazuhFW:~/.../api/test/integration(temp-23463-upgrade-agent-ait)$ pytest test_agent_PUT_endpoints.tavern.yaml 
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-7.3.1, pluggy-0.13.1
rootdir: /home/fdalmau/git/wazuh/api/test/integration
configfile: pytest.ini
plugins: aiohttp-1.0.4, trio-0.7.0, metadata-2.0.2, tavern-1.23.5, asyncio-0.18.1, html-2.1.1
asyncio: mode=auto
collected 10 items                                                                                                                                                                                        

test_agent_PUT_endpoints.tavern.yaml ..........                                                                                                                                                     [100%]

============================================================================================ warnings summary =============================================================================================
../../../../../venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================================================== 10 passed, 1 warning in 1156.37s (0:19:16) ================================================================================

`4.7.5`

Adding the same changes to Wazuh 4.7.5 maintains the same behavior.

(integrationtest-env) fdalmau@wazuhFW:~/.../api/test/integration(enhancement/23463-ait-upgrade-delay)$ pytest test_agent_PUT_endpoints.tavern.yaml 
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-7.3.1, pluggy-0.13.1
rootdir: /home/fdalmau/git/wazuh/api/test/integration
configfile: pytest.ini
plugins: aiohttp-1.0.4, trio-0.7.0, metadata-2.0.2, tavern-1.23.5, asyncio-0.18.1, html-2.1.1
asyncio: mode=auto
collected 10 items                                                                                                                                                                                        

test_agent_PUT_endpoints.tavern.yaml ..........                                                                                                                                                     [100%]

============================================================================================ warnings summary =============================================================================================
../../../../../venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================================================== 10 passed, 1 warning in 1182.80s (0:19:42) ================================================================================
(integrationtest-env) fdalmau@wazuhFW:~/.../api/test/integration(enhancement/23463-ait-upgrade-delay)$ pytest test_agent_PUT_endpoints.tavern.yaml --nobuild
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-7.3.1, pluggy-0.13.1
rootdir: /home/fdalmau/git/wazuh/api/test/integration
configfile: pytest.ini
plugins: aiohttp-1.0.4, trio-0.7.0, metadata-2.0.2, tavern-1.23.5, asyncio-0.18.1, html-2.1.1
asyncio: mode=auto
collected 10 items                                                                                                                                                                                        

test_agent_PUT_endpoints.tavern.yaml ..........                                                                                                                                                     [100%]

============================================================================================ warnings summary =============================================================================================
../../../../../venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642
  /home/fdalmau/venv/integrationtest-env/lib/python3.8/site-packages/_pytest/nodes.py:642: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================ 10 passed, 1 warning in 677.66s (0:11:17) ================================================================================

fdalmaup added the team/framework label May 16, 2024

Selutario added type/bug Something isn't working level/task and removed team/framework labels May 16, 2024

fdalmaup self-assigned this May 17, 2024

fdalmaup mentioned this issue May 20, 2024

Add upgrade_delay parameter to upgrade task creation test #23521

Merged

fdalmaup linked a pull request May 20, 2024 that will close this issue

Add upgrade_delay parameter to upgrade task creation test #23521

Merged

Selutario closed this as completed May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Investigate AIT errors for `PUT /agents/upgrade_custom` test #23463

Investigate AIT errors for `PUT /agents/upgrade_custom` test #23463

fdalmaup commented May 16, 2024 •

edited

fdalmaup commented May 17, 2024 •

edited

GGP1 commented May 17, 2024 •

edited

fdalmaup commented May 20, 2024 •

edited

Investigate AIT errors for PUT /agents/upgrade_custom test #23463

Investigate AIT errors for PUT /agents/upgrade_custom test #23463

Comments

fdalmaup commented May 16, 2024 • edited

Description

Checks

fdalmaup commented May 17, 2024 • edited

Issue Update

Conclusion

GGP1 commented May 17, 2024 • edited

Review

fdalmaup commented May 20, 2024 • edited

Issue Update

v4.7.4

4.7.5

Investigate AIT errors for `PUT /agents/upgrade_custom` test #23463

Investigate AIT errors for `PUT /agents/upgrade_custom` test #23463

fdalmaup commented May 16, 2024 •

edited

fdalmaup commented May 17, 2024 •

edited

GGP1 commented May 17, 2024 •

edited

fdalmaup commented May 20, 2024 •

edited

`v4.7.4`

`4.7.5`