Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module retrieved from URL without extension fails #22421

Open
peakschris opened this issue May 16, 2024 · 1 comment
Open

module retrieved from URL without extension fails #22421

peakschris opened this issue May 16, 2024 · 1 comment
Labels
help wanted Someone outside the Bazel team could own this P3 We're not considering working on this, but happy to review a PR. (No assignee) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@peakschris
Copy link

peakschris commented May 16, 2024

Description of the bug:

If a module is retrieved from a URL that does not have an archive extension, then the build fails. The URL can be set in a custom module definition, or with archive_override, or via a rewrite in downloader.cfg, all these cases fail.

For example, retrieving from a corporate cache:

bazel_dep(name = "rules_java", version = "7.5.0")
archive_override(
    module_name = "rules_java",
    urls = ["https://artifacts.ourcompany.com/artifactory/api/vcs/downloadTag/github/bazelbuild/rules_java/7.5.0?ext=tar.gz"]
)

Results in output:

D:\workspace>bazel build //src/build/out:all --config=offline2
INFO: Repository rules_java~ instantiated at:
  <builtin>: in <toplevel>
Repository rule http_archive defined at:
  D:/b/nag22qmp/external/bazel_tools/tools/build_defs/repo/http.bzl:391:31: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_java~':
   Traceback (most recent call last):
        File "D:/b/nag22qmp/external/bazel_tools/tools/build_defs/repo/http.bzl", line 136, column 45, in _http_archive_impl
                download_info = ctx.download_and_extract(
Error in download_and_extract: Expected a file with a .zip, .jar, .war, .aar, .tar, .tar.gz, .tgz, .tar.xz, .txz, .tar.zst, .tzst, .tar.bz2, .tbz, .ar or .deb suffix (got D:/b/nag22qmp/external/rules_java~/temp9089459162989031511/7.5.0)
ERROR: <builtin>: fetching http_archive rule //:rules_java~: Traceback (most recent call last):
        File "D:/b/nag22qmp/external/bazel_tools/tools/build_defs/repo/http.bzl", line 136, column 45, in _http_archive_impl
                download_info = ctx.download_and_extract(
Error in download_and_extract: Expected a file with a .zip, .jar, .war, .aar, .tar, .tar.gz, .tgz, .tar.xz, .txz, .tar.zst, .tzst, .tar.bz2, .tbz, .ar or .deb suffix (got D:/b/nag22qmp/external/rules_java~/temp9089459162989031511/7.5.0)
ERROR: Error computing the main repository mapping: error during computation of main repo mapping: Expected a file with a .zip, .jar, .war, .aar, .tar, .tar.gz, .tgz, .tar.xz, .txz, .tar.zst, .tzst, .tar.bz2, .tbz, .ar or .deb suffix (got D:/b/nag22qmp/external/rules_java~/temp9089459162989031511/7.5.0)

download_and_extract takes an expected suffix, but ideally any fix would support URL rewrites to URLs without extensions in downloader.cfg. When such URLs are retrieved from artifactory, they come back with these headers:

x-artifactory-filename: rules_java-7.5.0.tar.gz
content-type: application/x-gzip
content-disposition: attachment; filename="rules_java-7.5.0.tar.gz"

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Put an archive on a server with no extension, supply as archive_override (see above)

Which operating system are you running Bazel on?

Windows

What is the output of bazel info release?

release 7.2.0rc1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@github-actions github-actions bot added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label May 16, 2024
@peakschris
Copy link
Author

Note, I have found a workaround to this, by using a 'generic' mirror rather than 'vcs' mirror in Artifactory. This results in filenames that are the same as original. Feel free to close or leave open if a fix would still be useful to others.

@meteorcloudy meteorcloudy added type: feature request P2 We'll consider working on this in future. (Assignee optional) help wanted Someone outside the Bazel team could own this P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed type: bug untriaged P2 We'll consider working on this in future. (Assignee optional) labels May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Someone outside the Bazel team could own this P3 We're not considering working on this, but happy to review a PR. (No assignee) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

No branches or pull requests

5 participants