Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: add cacheing #1639

Open
dabrahams opened this issue Mar 3, 2024 · 1 comment
Open

Feature request: add cacheing #1639

dabrahams opened this issue Mar 3, 2024 · 1 comment

Comments

@dabrahams
Copy link

I have to check out the same large repository (LLVM) in two different jobs and many matrix configurations in my workflow, and it's nearly always checked out at the same commit. If I could use a cache for that repo, it would speed up CI considerably.
git ls-remote https://github.com/llvm/llvm-project.git -t llvmorg-$LLVM_VERSION | cut -f 1 could be used as a cache key. I may be able to figure out how to get this to work on my own, but it would be awesome if the functionality were built in.

Thanks!

@davhdavh
Copy link

davhdavh commented Jun 7, 2024

For download caching to make any sense, you need to somehow have it "closer" to where you need it. So for a public build runner, that gets reset every time and only have access to anything via internet, downloading it from the "cache" or downloading it from the source is about the same thing.

So you need to have a self-hosted action runner where you have a drive/share/folder that survives between builds.

    - name: Update git cache
      run: |
          set -e
          mkdir -p $REPO_CACHE/${{ github.repository }}
          if [ ! -f "$REPO_CACHE/${{ github.repository }}/HEAD" ]; then
            rm -rf $REPO_CACHE/${{ github.repository }}
            git -c credential.helper='!f() { sleep 1; echo "username=${{github.repository_owner}}"; echo "password=$GITHUB_TOKEN"; }; f' clone --mirror https://github.com/${{github.repository}} $REPO_CACHE/${{ github.repository }}
          fi
          pushd $REPO_CACHE/${{ github.repository }}
          git -c credential.helper='!f() { sleep 1; echo "username=${{github.repository_owner}}"; echo "password=$GITHUB_TOKEN"; }; f' fetch origin
          popd
      env:
        GITHUB_TOKEN: ${{ github.token }}

    - name: Checkout
      run: |
          set -e
          git clone --no-checkout $REPO_CACHE/${{ github.repository }} .
          git checkout --progress --force ${{ github.sha }}

For obvious reasons this is a major security issue if you don't trust all jobs running on your runner to not have read access to all other repos implicitly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants