[Tests] Azure disk tier test fail

[Michaelvll]

test_azure_disk_tier and test_azure_best_tier_failover failed on the disk type check: az resource list --tag ray-cluster-name=t-azure-best-tier-fai-4u-97-0354 --query "[?type=='Microsoft.Compute/disks'].sku.name" --output tsv | grep Standard_LRS

az --version
azure-cli                         2.56.0 *

core                              2.56.0 *
telemetry                          1.1.0

Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2

Python location '/opt/conda/envs/sky/bin/python3.10'
Extensions directory '/home/gcpuser/.azure/cliextensions'

Python (Linux) 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0]

Legal docs and information: aka.ms/AzureCliLegal

[romilbhardwaj]

I also observed this recently when I was running smoke tests from GCP machine. FWIW, the same tests passed when I ran them from my laptop.

My laptop versions:

azure-cli                         2.45.0 *

core                              2.45.0 *
telemetry                          1.0.8 *

Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1

Python location '/Users/romilb/tools/anaconda3/bin/python'
Extensions directory '/Users/romilb/.azure/cliextensions'

Python (Darwin) 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33)
[Clang 13.0.1 ]

Legal docs and information: aka.ms/AzureCliLegal

[cblmemo]

The following version works for me. Lemme try if 2.56.0 works

$ az --version           
azure-cli                         2.57.0 *

core                              2.57.0 *
telemetry                          1.1.0

Dependencies:
msal                              1.26.0
azure-mgmt-resource             23.1.0b2

Python location '/home/memory/install/miniconda3/envs/sky/bin/python'
Extensions directory '/home/memory/.azure/cliextensions'

Python (Linux) 3.9.18 (main, Sep 11 2023, 13:41:44) 
[GCC 11.2.0]

Legal docs and information: aka.ms/AzureCliLegal

[cblmemo]

Humm, i tried with same az version as @Michaelvll , and at least the az resource list --tag ray-cluster-name=t-azure-best-tier-fai-4u-97-0354 --query "[?type=='Microsoft.Compute/disks'].sku.name" --output tsv works for me. I'm running smoke test for the version now. Could you share your error message at the same time?

My version:

az --version
azure-cli                         2.56.0 *

core                              2.56.0 *
telemetry                          1.1.0

Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2

Python location '/home/memory/install/miniconda3/envs/sky-az-test/bin/python'
Extensions directory '/home/memory/.azure/cliextensions'

Python (Linux) 3.9.19 (main, Mar 21 2024, 17:11:28) 
[GCC 11.2.0]

Legal docs and information: aka.ms/AzureCliLegal

[cblmemo]

Update: I passed the smoke test using this version as well..

D 05-02 22:00:10 skypilot_config.py:144] Using config path: /home/memory/.sky/config.yaml
D 05-02 22:00:10 skypilot_config.py:148] Config loaded:
D 05-02 22:00:10 skypilot_config.py:148] {'serve': {'controller': {'resources': {'cloud': 'aws', 'cpus': 4}}},
D 05-02 22:00:10 skypilot_config.py:148]  'spot': {'controller': {'resources': {'cloud': 'aws', 'cpus': 4}}}}
D 05-02 22:00:10 skypilot_config.py:158] Config syntax check passed.
[azure-disk-tier-low] Test started. Log: less /tmp/azure-disk-tier-low-yc4yrw6e.log
[azure-best-tier-failover] Test started. Log: less /tmp/azure-best-tier-failover-4d842sot.log
[azure-disk-tier-low] �[32mPassed�[0m.
[azure-disk-tier-low] Log: less /tmp/azure-disk-tier-low-yc4yrw6e.log
[azure-disk-tier-low] 
[azure-best-tier-failover] �[32mPassed�[0m.
[azure-best-tier-failover] Log: less /tmp/azure-best-tier-failover-4d842sot.log
[azure-best-tier-failover] 
[azure-disk-tier-medium] Test started. Log: less /tmp/azure-disk-tier-medium-qouw2ufd.log
bringing up nodes...
bringing up nodes...

.[azure-disk-tier-medium] �[32mPassed�[0m.
[azure-disk-tier-medium] Log: less /tmp/azure-disk-tier-medium-qouw2ufd.log
[azure-disk-tier-medium] 
[azure-disk-tier-best] Test started. Log: less /tmp/azure-disk-tier-best-2btm4o31.log
[azure-disk-tier-best] �[32mPassed�[0m.
[azure-disk-tier-best] Log: less /tmp/azure-disk-tier-best-2btm4o31.log
[azure-disk-tier-best] 
.
2 passed, 3284 warnings in 3394.62s (0:56:34)