Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cs complete for snapsots not working like resolve or bootstrap #1700

Open
ckipp01 opened this issue May 2, 2020 · 2 comments · May be fixed by #2655
Open

cs complete for snapsots not working like resolve or bootstrap #1700

ckipp01 opened this issue May 2, 2020 · 2 comments · May be fixed by #2655

Comments

@ckipp01
Copy link
Contributor

ckipp01 commented May 2, 2020

Is there a way to do the following:

cs complete -r sonatype:snapshots org.scalameta:metals_2.12:

This only returns stable version rather than the snapshots I'd expect.

However if I target a snapshot directly with resolve, it does work.

cs resolve -r sonatype:snapshots org.scalameta:metals_2.12:0.8.4+127-65087a2b-SNAPSHOT

If resolve is able to resolve this snapshot, I sort of expected complete to be able to pick up on all the published snapshots. Is there a way to do this that I'm missing by chance?
@ckipp01
Copy link
Contributor Author

ckipp01 commented Apr 15, 2021

Following up on this from Gitter today. I realized I actually had an issue created for this a year ago 😬 .

@ckipp01 ckipp01 mentioned this issue Apr 22, 2021
Merged
@keynmol
Copy link
Contributor

keynmol commented Sep 3, 2021 •
edited

After a lot of printlns, I think it's a combination of how coursier completion works + how sonatype snapshots are setup differently.

Main reason can be demonstrated easily:

➜  ~ curl -s -L https://oss.sonatype.org/content/repositories/releases/ | head
<!DOCTYPE html>
<html>

<head>
	<title>Central Repository: </title>
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<style>
body {
	background: #fff;
}

➜  ~ curl -s -L https://oss.sonatype.org/content/repositories/snapshots/ | head
Directory listing forbidden%

And it seems that coursier gradually peels off the artfiact parts until it gets to the root (i.e. after removing org), until it needs to download (and potentially cache) the listing for /snapshots root on Sonatype. But that fails.

./mill -i -w cli.run complete --cache force --no-default -r sonatype:snapshots org.scalameta:metals_2.12: 
List(LocalUpdateChanging, LocalOnly, Update)
Processing MavenRepository(https://oss.sonatype.org/content/repositories/snapshots, None, None, true, true) with MavenComplete(MavenRepository(https://oss.sonatype.org/content/repositories/snapshots, None, None, true, true), coursier.cache.FileCache$$Lambda$845/0x0000000800422040@5eac003a, coursier.util.PlatformTaskCompanion$$anon$2@5a938f37)
Input Ver(org.scalameta:metals_2.12, org.scalameta:metals_2.12:, 26)
DIR URL: https://oss.sonatype.org/content/repositories/snapshots/org/
Artifact: Artifact(https://oss.sonatype.org/content/repositories/snapshots/org/.links, Map(), Map(), true, false, None)
Artifact(https://oss.sonatype.org/content/repositories/snapshots/org/, Map(), Map(), true, false, None)
true
Checking org Org(org.scalameta)
DIR URL: https://oss.sonatype.org/content/repositories/snapshots/
Artifact: Artifact(https://oss.sonatype.org/content/repositories/snapshots/.links, Map(), Map(), true, false, None)
Artifact(https://oss.sonatype.org/content/repositories/snapshots/, Map(), Map(), true, false, None)
true
Checking org Org(org)
Downloader for Artifact(https://oss.sonatype.org/content/repositories/snapshots/, Map(), Map(), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/snapshots/.directory for url https://oss.sonatype.org/content/repositories/snapshots/
true -- true -- LocalUpdateChanging
Artifact(https://oss.sonatype.org/content/repositories/snapshots/, Map(), Map(), true, false, None)
true
Downloader for Artifact(https://oss.sonatype.org/content/repositories/snapshots/, Map(), Map(), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/snapshots/.directory for url https://oss.sonatype.org/content/repositories/snapshots/
true -- true -- LocalOnly
Artifact(https://oss.sonatype.org/content/repositories/snapshots/, Map(), Map(), true, false, None)
true
Downloader for Artifact(https://oss.sonatype.org/content/repositories/snapshots/, Map(), Map(), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/snapshots/.directory for url https://oss.sonatype.org/content/repositories/snapshots/
true -- true -- Update
Needs update: true
DOWNLOADING https://oss.sonatype.org/content/repositories/snapshots/

In comparison, when using sonatype:releases:

./mill -i -w cli.run complete --cache force --no-default -r sonatype:releases org.scalameta:metals_2.12: 
List(LocalUpdateChanging, LocalOnly, Update)
Processing MavenRepository(https://oss.sonatype.org/content/repositories/releases, None, None, true, true) with MavenComplete(MavenRepository(https://oss.sonatype.org/content/repositories/releases, None, None, true, true), coursier.cache.FileCache$$Lambda$845/0x0000000800422040@49a05c35, coursier.util.PlatformTaskCompanion$$anon$2@4d9fc1d9)
Input Ver(org.scalameta:metals_2.12, org.scalameta:metals_2.12:, 26)
DIR URL: https://oss.sonatype.org/content/repositories/releases/org/
Artifact: Artifact(https://oss.sonatype.org/content/repositories/releases/org/.links, Map(), Map(), true, false, None)
Artifact(https://oss.sonatype.org/content/repositories/releases/org/, Map(), Map(), true, false, None)
true
Checking org Org(org.scalameta)
DIR URL: https://oss.sonatype.org/content/repositories/releases/
Artifact: Artifact(https://oss.sonatype.org/content/repositories/releases/.links, Map(), Map(), true, false, None)
Artifact(https://oss.sonatype.org/content/repositories/releases/, Map(), Map(), true, false, None)
true
Checking org Org(org)
Downloader for Artifact(https://oss.sonatype.org/content/repositories/releases/, Map(), Map(), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/releases/.directory for url https://oss.sonatype.org/content/repositories/releases/
true -- true -- LocalUpdateChanging
Needs update: false
Result of org: Right(Result(Org(org), Vector(org, org.elasticsearch, org.elasticsearch.client, org.elasticsearch.distribution.integ-test-zip, org.elasticsearch.distribution.zip, org.elasticsearch.gradle, org.elasticsearch.plugin, org.elasticsearch.test)))
Result of check: true
Downloader for Artifact(https://oss.sonatype.org/content/repositories/releases/org/, Map(), Map(), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/releases/org/.directory for url https://oss.sonatype.org/content/repositories/releases/org/
true -- true -- LocalUpdateChanging
Needs update: false
Result of org: Right(Result(Org(org.scalameta), Vector(org.scalameta)))
Result of check: true
Artifact: Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/.links, Map(), Map(), true, false, None)
Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/, Map(), Map(), true, false, None)
true
Downloader for Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/, Map(), Map(), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/releases/org/scalameta/.directory for url https://oss.sonatype.org/content/repositories/releases/org/scalameta/
true -- true -- LocalUpdateChanging
Needs update: false
org.scalameta:metals_2.12 in https://oss.sonatype.org/content/repositories/releases
Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml, Map(MD5 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.md5, SHA-1 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.sha1, SHA-256 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.sha256), Map(cache-errors -> Artifact(, Map(), Map(), false, false, None), sig -> Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc, Map(MD5 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc.md5, SHA-1 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc.sha1, SHA-256 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc.sha256), Map(), true, true, None)), true, false, None)
false
Downloader for Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml, Map(MD5 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.md5, SHA-1 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.sha1, SHA-256 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.sha256), Map(cache-errors -> Artifact(, Map(), Map(), false, false, None), sig -> Artifact(https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc, Map(MD5 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc.md5, SHA-1 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc.sha1, SHA-256 -> https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml.asc.sha256), Map(), true, true, None)), true, false, None)
Local file: force/https/oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml for url https://oss.sonatype.org/content/repositories/releases/org/scalameta/metals_2.12/maven-metadata.xml
true -- true -- LocalUpdateChanging
Needs update: false
VERSIONS: Right(Result(Ver(org.scalameta:metals_2.12, org.scalameta:metals_2.12:, 26), List(0.1.0, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.11, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.5.0-M1, 0.5.0, 0.5.1, 0.5.2, 0.6.0, 0.6.1, 0.7.0, 0.7.2, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.9.9, 0.9.10, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.5, 0.10.6-M1)))
(MavenRepository(https://oss.sonatype.org/content/repositories/releases, None, None, true, true),Right(List()))

You can see that with releases, it succeeded in downloading https://oss.sonatype.org/content/repositories/releases/.links (which is a special filename coursier creates to indicate that webpage needs to be scraped), and proceeded to work upwards - downloading org/, then scalameta/, etc.

This is why I think resolution of snapshots doesn't work

@keynmol keynmol mentioned this issue Sep 3, 2021
Closed
@ckipp01 ckipp01 mentioned this issue Dec 28, 2021
Open
@jackkoenig jackkoenig linked a pull request Jan 7, 2023 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove recursion from Complete.hasOrg jackkoenig/coursier
2 participants
@keynmol @ckipp01