Extend SCC construction to enable extra functionality #125069

amandasystems · 2024-05-13T12:06:03Z

Do YOU feel like your SCC construction doesn't do enough? Then I have a patch for you! SCCs can now do everything! Well, almost.

This patch has been extracted from #123720. It specifically enhances
Sccs to allow tracking arbitrary commutative properties (think min/max mappings on nodes vs arbitrary closures) of strongly connected components, including

reachable values (max/min)
SCC-internal values (max/min)

This helps with among other things universe computation. We can now identify
SCC universes as a reasonably straightforward "find max/min" operation during SCC construction. This is also included in this patch.

It's also more or less zero-cost; don't use the new features, don't pay for them.

This commit also vastly extends the documentation of the SCCs module, which I had a very hard time following. It may or may not have gotten easier to read for someone else.

I believe this logic can also be used in leak check, but haven't checked. Ha. ha. Ha.

rustbot · 2024-05-13T12:06:12Z

r? @pnkfelix

rustbot has assigned @pnkfelix.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

amandasystems · 2024-05-13T12:21:30Z

r? Lqd

lqd · 2024-05-13T13:08:22Z

@bors try @rust-timer queue

As discussed on zulip, until I have more time let's also temporarily redirect to niko for review or reassignment, and discuss at this week's meeting: r? nikomatsakis

Extend SCC construction to enable extra functionality Do YOU feel like your SCC construction doesn't do enough? Then I have a patch for you! SCCs can now do *everything*! Well, almost. This patch has been extracted from rust-lang#123720. It specifically enhances `Sccs` to allow tracking arbitrary commutative properties (think min/max mappings on nodes vs arbitrary closures) of strongly connected components, including - reachable values (max/min) - SCC-internal values (max/min) This helps with among other things universe computation. We can now identify SCC universes as a reasonably straightforward "find max/min" operation during SCC construction. This is also included in this patch. It's also more or less zero-cost; don't use the new features, don't pay for them. This commit also vastly extends the documentation of the SCCs module, which I had a very hard time following. It may or may not have gotten easier to read for someone else. I believe this logic can also be used in leak check, but haven't checked. Ha. ha. Ha.

bors · 2024-05-13T13:09:33Z

⌛ Trying commit 609a202 with merge 6c7f0a6...

bors · 2024-05-13T14:47:07Z

☀️ Try build successful - checks-actions
Build commit: 6c7f0a6 (6c7f0a6914780a8bd5240c26e668f7b0625b66dc)

rust-timer · 2024-05-13T16:03:36Z

Finished benchmarking commit (6c7f0a6): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.6%]	45
Regressions ❌ (secondary)	1.4%	[0.1%, 4.0%]	32
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[0.2%, 1.6%]	45

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.8%	[2.4%, 3.2%]	2
Regressions ❌ (secondary)	2.2%	[2.2%, 2.2%]	2
Improvements ✅ (primary)	-2.6%	[-2.6%, -2.6%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.0%	[-2.6%, 3.2%]	3

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.9%	[1.6%, 2.1%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.9%	[-2.9%, -2.9%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 675.291s -> 676.941s (0.24%)
Artifact size: 315.98 MiB -> 316.29 MiB (0.10%)

Kobzol · 2024-05-17T13:19:27Z

@bors try @rust-timer queue

bors · 2024-05-17T13:25:36Z

⌛ Trying commit c510e73 with merge 1805d8a...

Extend SCC construction to enable extra functionality Do YOU feel like your SCC construction doesn't do enough? Then I have a patch for you! SCCs can now do *everything*! Well, almost. This patch has been extracted from rust-lang#123720. It specifically enhances `Sccs` to allow tracking arbitrary commutative properties (think min/max mappings on nodes vs arbitrary closures) of strongly connected components, including - reachable values (max/min) - SCC-internal values (max/min) This helps with among other things universe computation. We can now identify SCC universes as a reasonably straightforward "find max/min" operation during SCC construction. This is also included in this patch. It's also more or less zero-cost; don't use the new features, don't pay for them. This commit also vastly extends the documentation of the SCCs module, which I had a very hard time following. It may or may not have gotten easier to read for someone else. I believe this logic can also be used in leak check, but haven't checked. Ha. ha. Ha.

amandasystems · 2024-05-17T15:00:46Z

My guess is 0.3% primary regressions and 0.84% in secondary, let's see how wrong I am!

Also, you can't edit on GitHub so there is no way for me to remove this if I'm enormously off. Too bad!

lqd · 2024-05-17T16:40:27Z

@rust-timer build 1805d8a

rust-timer · 2024-05-17T18:41:14Z

Finished benchmarking commit (1805d8a): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.8%]	23
Regressions ❌ (secondary)	1.5%	[0.1%, 3.9%]	20
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[0.2%, 0.8%]	23

Max RSS (memory usage)

Results (primary -0.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.3%	[1.3%, 1.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-2.4%, -0.2%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.4%	[-2.4%, 1.3%]	3

Cycles

Results (primary -0.2%, secondary -1.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	7.2%	[7.2%, 7.2%]	1
Regressions ❌ (secondary)	2.0%	[1.8%, 2.2%]	5
Improvements ✅ (primary)	-2.1%	[-4.3%, -0.8%]	4
Improvements ✅ (secondary)	-2.6%	[-3.8%, -2.0%]	10
All ❌✅ (primary)	-0.2%	[-4.3%, 7.2%]	5

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 669.422s -> 671.224s (0.27%)
Artifact size: 316.04 MiB -> 316.26 MiB (0.07%)

nikomatsakis · 2024-06-12T09:33:15Z

Actually that was slightly premature. Let me finish the read :)

@bors r-

nikomatsakis

r=me modulo nits

I love all the comments you added! 💜

compiler/rustc_borrowck/src/region_infer/mod.rs

compiler/rustc_data_structures/src/graph/scc/mod.rs

This patch has been extracted from rust-lang#123720. It specifically enhances `Sccs` to allow tracking arbitrary commutative properties of SCCs, including - reachable values (max/min) - SCC-internal values (max/min) This helps with among other things universe computation: we can now identify SCC universes as a straightforward "find max/min" operation during SCC construction. It's also more or less zero-cost; don't use the new features, don't pay for them. This commit also vastly extends the documentation of the SCCs module, which I had a very hard time following.

Note that this changes no executing code. The change is 100% in documentation.

This version shaves off ca 2% of the cycles in my experiments and makes the control flow easier to follow for me and hopefully others, including the compiler. Someone gave me a working profiler and by God I'm using it.

This shaves off ca 6% of the cycles in `start_walk_from()` in my experiments.

Apparently this interferes with inlining and murders performance on `wg-grammar`.

@lqd

This commit addresses @lqd's code review and resurrects a lost comment and removes some dead code.

In terms of code organisation, this is a lot cleaner and allows tighter access modifiers.

amandasystems · 2024-06-12T14:04:11Z

Rebased!

jackh726 · 2024-06-12T16:22:42Z

@bors r=nikomatsakis

bors · 2024-06-12T16:22:45Z

📌 Commit d63708b has been approved by nikomatsakis

It is now in the queue for this repository.

bors · 2024-06-12T23:15:35Z

⌛ Testing commit d63708b with merge 8cf5101...

bors · 2024-06-13T01:56:50Z

☀️ Test successful - checks-actions
Approved by: nikomatsakis
Pushing 8cf5101 to master...

rust-timer · 2024-06-13T03:13:59Z

Finished benchmarking commit (8cf5101): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.3%]	6
Regressions ❌ (secondary)	0.5%	[0.1%, 1.0%]	10
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	2
All ❌✅ (primary)	0.2%	[0.2%, 0.3%]	6

Max RSS (memory usage)

Results (primary -3.2%, secondary -5.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.2%	[-4.4%, -2.0%]	2
Improvements ✅ (secondary)	-5.0%	[-5.0%, -5.0%]	1
All ❌✅ (primary)	-3.2%	[-4.4%, -2.0%]	2

Cycles

Results (secondary -1.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.9%	[2.0%, 3.6%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-8.1%	[-9.1%, -7.5%]	3
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 671.52s -> 672.648s (0.17%)
Artifact size: 320.35 MiB -> 320.29 MiB (-0.02%)

rustbot assigned pnkfelix May 13, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 13, 2024

rustbot assigned lqd and unassigned pnkfelix May 13, 2024

This comment has been minimized.

Sign in to view

rustbot assigned nikomatsakis and unassigned lqd May 13, 2024

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 13, 2024

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels May 13, 2024

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 17, 2024

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 17, 2024

amandasystems force-pushed the scc-refactor branch from 2be0958 to e1204fe Compare May 20, 2024 11:05

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jun 12, 2024

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 12, 2024

nikomatsakis reviewed Jun 12, 2024

View reviewed changes

compiler/rustc_borrowck/src/region_infer/mod.rs Outdated Show resolved Hide resolved

compiler/rustc_data_structures/src/graph/scc/mod.rs Outdated Show resolved Hide resolved

amandasystems added 12 commits June 12, 2024 15:47

Documentation fixes

d2a0176

Simplify path compression logic

905db03

Formatting, weird because I just did that

582c613

Docstring for for Annotation

14c10ec

Note that this changes no executing code. The change is 100% in documentation.

Slightly faster version of find_state

b1add7b

This version shaves off ca 2% of the cycles in my experiments and makes the control flow easier to follow for me and hopefully others, including the compiler. Someone gave me a working profiler and by God I'm using it.

Remove a few unnecessary constructions

aee8462

This shaves off ca 6% of the cycles in `start_walk_from()` in my experiments.

Roll back a few #[instrument]

7782d20

Apparently this interferes with inlining and murders performance on `wg-grammar`.

Resurrect some comments, remove a pub

e6eb63d

This commit addresses @lqd's code review and resurrects a lost comment and removes some dead code.

Revise documentation after @lqd's comments

3bdcb9d

Move RegionTracker to region_infer

2e1e119

In terms of code organisation, this is a lot cleaner and allows tighter access modifiers.

Address code review comments on the comments

d63708b

amandasystems force-pushed the scc-refactor branch from e754d68 to d63708b Compare June 12, 2024 14:03

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 12, 2024

bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 13, 2024

bors merged commit 8cf5101 into rust-lang:master Jun 13, 2024
7 checks passed

rustbot added this to the 1.81.0 milestone Jun 13, 2024

bors mentioned this pull request Jun 13, 2024

rustfmt use declarations #125443

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend SCC construction to enable extra functionality #125069

Extend SCC construction to enable extra functionality #125069

amandasystems commented May 13, 2024

rustbot commented May 13, 2024

amandasystems commented May 13, 2024

lqd commented May 13, 2024

This comment has been minimized.

bors commented May 13, 2024

bors commented May 13, 2024

This comment has been minimized.

rust-timer commented May 13, 2024

Kobzol commented May 17, 2024

This comment has been minimized.

bors commented May 17, 2024

amandasystems commented May 17, 2024 •

edited

lqd commented May 17, 2024

This comment has been minimized.

rust-timer commented May 17, 2024

nikomatsakis commented Jun 12, 2024

nikomatsakis left a comment

amandasystems commented Jun 12, 2024

jackh726 commented Jun 12, 2024

bors commented Jun 12, 2024

bors commented Jun 12, 2024

bors commented Jun 13, 2024

rust-timer commented Jun 13, 2024

Extend SCC construction to enable extra functionality #125069

Extend SCC construction to enable extra functionality #125069

Conversation

amandasystems commented May 13, 2024

rustbot commented May 13, 2024

amandasystems commented May 13, 2024

lqd commented May 13, 2024

This comment has been minimized.

bors commented May 13, 2024

bors commented May 13, 2024

This comment has been minimized.

rust-timer commented May 13, 2024

Overall result: ❌ regressions - ACTION NEEDED

Kobzol commented May 17, 2024

This comment has been minimized.

bors commented May 17, 2024

amandasystems commented May 17, 2024 • edited

lqd commented May 17, 2024

This comment has been minimized.

rust-timer commented May 17, 2024

Overall result: ❌ regressions - ACTION NEEDED

nikomatsakis commented Jun 12, 2024

nikomatsakis left a comment

Choose a reason for hiding this comment

amandasystems commented Jun 12, 2024

jackh726 commented Jun 12, 2024

bors commented Jun 12, 2024

bors commented Jun 12, 2024

bors commented Jun 13, 2024

rust-timer commented Jun 13, 2024

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

amandasystems commented May 17, 2024 •

edited