Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conda env creation refers depreciated URL #178

Closed
KeitaW opened this issue Mar 5, 2024 · 2 comments
Closed

Conda env creation refers depreciated URL #178

KeitaW opened this issue Mar 5, 2024 · 2 comments
Labels

Comments

@KeitaW
Copy link
Contributor

KeitaW commented Mar 5, 2024

0.create_conda_env.sh in 10.FSDP refers deprecated url https://aws-ml-conda-ec2.s3.us-west-2.amazonaws.com.

$ bash 0.create_conda_env.sh 
+ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
--2024-03-05 23:15:13--  https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.131.3, 104.16.130.3, 2606:4700::6810:8303, ...
Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.131.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 144041912 (137M) [application/octet-stream]
Saving to: ‘Miniconda3-latest-Linux-x86_64.sh’

Miniconda3-latest-Linux-x86_64.sh                            100%[============================================================================================================================================>] 137.37M   209MB/s    in 0.7s    

2024-03-05 23:15:13 (209 MB/s) - ‘Miniconda3-latest-Linux-x86_64.sh’ saved [144041912/144041912]

+ chmod +x Miniconda3-latest-Linux-x86_64.sh
+ ./Miniconda3-latest-Linux-x86_64.sh -b -f -p ./miniconda3
PREFIX=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3
Unpacking payload ...
                                                                                                                                                                                                                                                  
Installing base environment...


Downloading and Extracting Packages:


Downloading and Extracting Packages:

Preparing transaction: done
Executing transaction: done
installation finished.
+ source ./miniconda3/bin/activate
++ _CONDA_ROOT=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3
++ . /fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/etc/profile.d/conda.sh
+++ export CONDA_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda
+++ CONDA_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda
+++ export _CE_M=
+++ _CE_M=
+++ export _CE_CONDA=
+++ _CE_CONDA=
+++ export CONDA_PYTHON_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/python
+++ CONDA_PYTHON_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/python
+++ '[' -z x ']'
++ conda activate
++ local cmd=activate
++ case "$cmd" in
++ __conda_activate activate
++ '[' -n '' ']'
++ local ask_conda
+++ PS1=
+++ __conda_exe shell.posix activate
+++ /fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda shell.posix activate
++ ask_conda='PS1='\''(base) '\''
export PATH='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin:/opt/conda/condabin:/home/ubuntu/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/remote-cli:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/opt/amazon/openmpi/bin:/opt/amazon/efa/bin:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/aws/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/cfn_bootstrap_virtualenv/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/awsbatch_virtualenv/bin:/opt/slurm/bin'\''
export CONDA_PREFIX='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3'\''
export CONDA_SHLVL='\''2'\''
export CONDA_DEFAULT_ENV='\''base'\''
export CONDA_PROMPT_MODIFIER='\''(base) '\''
export CONDA_PREFIX_1='\''/opt/conda'\''
export CONDA_EXE='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda'\''
export _CE_M='\'''\''
export _CE_CONDA='\'''\''
export CONDA_PYTHON_EXE='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/python'\'''
++ eval 'PS1='\''(base) '\''
export PATH='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin:/opt/conda/condabin:/home/ubuntu/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/remote-cli:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/opt/amazon/openmpi/bin:/opt/amazon/efa/bin:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/aws/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/cfn_bootstrap_virtualenv/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/awsbatch_virtualenv/bin:/opt/slurm/bin'\''
export CONDA_PREFIX='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3'\''
export CONDA_SHLVL='\''2'\''
export CONDA_DEFAULT_ENV='\''base'\''
export CONDA_PROMPT_MODIFIER='\''(base) '\''
export CONDA_PREFIX_1='\''/opt/conda'\''
export CONDA_EXE='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda'\''
export _CE_M='\'''\''
export _CE_CONDA='\'''\''
export CONDA_PYTHON_EXE='\''/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/python'\'''
+++ PS1='(base) '
+++ export PATH=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin:/opt/conda/condabin:/home/ubuntu/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/remote-cli:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/opt/amazon/openmpi/bin:/opt/amazon/efa/bin:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/aws/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/cfn_bootstrap_virtualenv/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/awsbatch_virtualenv/bin:/opt/slurm/bin
+++ PATH=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin:/opt/conda/condabin:/home/ubuntu/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/remote-cli:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/opt/amazon/openmpi/bin:/opt/amazon/efa/bin:/opt/conda/bin:/usr/local/cuda/bin:/usr/local/cuda/include:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/aws/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/cfn_bootstrap_virtualenv/bin:/opt/parallelcluster/pyenv/versions/3.9.17/envs/awsbatch_virtualenv/bin:/opt/slurm/bin
+++ export CONDA_PREFIX=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3
+++ CONDA_PREFIX=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3
+++ export CONDA_SHLVL=2
+++ CONDA_SHLVL=2
+++ export CONDA_DEFAULT_ENV=base
+++ CONDA_DEFAULT_ENV=base
+++ export 'CONDA_PROMPT_MODIFIER=(base) '
+++ CONDA_PROMPT_MODIFIER='(base) '
+++ export CONDA_PREFIX_1=/opt/conda
+++ CONDA_PREFIX_1=/opt/conda
+++ export CONDA_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda
+++ CONDA_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda
+++ export _CE_M=
+++ _CE_M=
+++ export _CE_CONDA=
+++ _CE_CONDA=
+++ export CONDA_PYTHON_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/python
+++ CONDA_PYTHON_EXE=/fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/python
++ __conda_hashr
++ '[' -n '' ']'
++ '[' -n '' ']'
++ hash -r
+ conda create -y -p ./pt_fsdp python=3.10
+ local cmd=create
+ case "$cmd" in
+ __conda_exe create -y -p ./pt_fsdp python=3.10
+ /fsx/ben-awsome-distributed-training/3.test_cases/10.FSDP/miniconda3/bin/conda create -y -p ./pt_fsdp python=3.10
Retrieving notices: ...working... done
Channels:
 - https://aws-ml-conda-ec2.s3.us-west-2.amazonaws.com
 - conda-forge
 - nvidia
 - pytorch
Platform: linux-64
Collecting package metadata (repodata.json): failed

UnavailableInvalidChannel: HTTP 404 NOT FOUND for channel  <https://aws-ml-conda-ec2.s3.us-west-2.amazonaws.com>

The channel is not accessible or is invalid.

You will need to adjust your conda configuration to proceed.
Use `conda config --show channels` to view your configuration's current state,
and use `conda config --show-sources` to view config file locations.
$ conda config --show-sources
==> /opt/conda/.condarc <==
channels:
  - conda-forge

==> /home/ubuntu/.condarc <==
auto_update_conda: False
channel_priority: strict
channels:
  - https://aws-ml-conda-ec2.s3.us-west-2.amazonaws.com
  - conda-forge
  - nvidia
  - pytorch
@KeitaW KeitaW linked a pull request Mar 5, 2024 that will close this issue
Copy link

github-actions bot commented Apr 5, 2024

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Apr 5, 2024
Copy link

github-actions bot commented Jun 4, 2024

This issue was closed because it has been inactive for 14 days since being marked as stale.

@github-actions github-actions bot closed this as completed Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant