You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the CI workflow is extremely slow (10-20 min average). It's not a big deal, but when you have to accumulate a push, a dependency update merge, another run for another dependency PR, another one for the second merge, and so on, it becomes very time-consuming and blocking for no real reason.
I say blocking because you have to sequentially wait for each one so that your release draft can actually be created, so you can finally publish it, wait for another PR workflow with the CHANGELOG.md update, and a final last one when you merge the changelog PR.
It's tedious, slow, and uses GH Actions bandwidth for practically nothing. I can sometimes waste 30-60min for a 5-line fix that I release alongside 2 Renovate PRs.
Proposed solution
The solution lies in 2 words: conditional execution.
I've used this technique in some of my recent projects, where I make my CI smart: it only runs the steps it needs to run depending on the changes I make.
I would skip build, test, inspectCode, and verify when no change is made to any gradle dependency/src code
I would skip test if there is no test folder
I would skip releaseDraft if a release with the same name has already been released
I would potentiallyskip test, inspectCode and verify if the change is only related to gradle dependencies
What would those changes mean?
Taking our same example above, we'd go from:
Before: 1 run for a 5-line fix, 1 run for DEPS_PR_1 getting rebased, 1 run for DEPS_PR_2 getting rebased, 1 run for DEPS_PR_1 getting merged, 1 run for DEPS_PR_2 getting rebased again, 1 run for DEPS_PR_2 getting merged, 1 run for CHANGELOG_PR getting created and 1 run for CHANGELOG_PR getting merged
To:
After: 1 run for a 5-line fix, 1 very short run for DEPS_PR_1 getting rebased, 1 very short run for DEPS_PR_2 getting rebased, 1 very short run for DEPS_PR_1 getting merged, 1 very short run for DEPS_PR_2 getting rebased again, 1 very short run for DEPS_PR_2 getting merged, 0 run for CHANGELOG_PR getting created and 0 run for CHANGELOG_PR getting merged, and no incorrect draft release to manually delete
Would that be a big deal?
With rough average estimations for a relatively medium codebase of:
build: 2 min
test: 2 min (even without any test)
inspectCode: 6 min
verify: 7 min
And being generous because Post Setup Gradle can be extremely slow, sometimes doubling the job's duration but that's not on you, we would go from ~72 min (8 * 9 min) to ~19 min (1 * 9 min + 5 * 2 min + 2 * 0s) of run for a casual workflow with hotfix releases and dependency management.
Additional actions improvements
If you want to go beyond, maybe improving the run duration of your own actions, especially ./gradlew runPluginVerifier and JetBrains/qodana-action, which are the slowest of all (maybe parallelizing stuff like IDE checks), it'd be amazing!
If you guys also have an idea of why and how Post Setup Gradle can take up to 4-5 min, I'd love to hear from you!
Additionally, I'd love to see Qodana reports in PRs not getting sent on closed/merged PRs.
Conclusion
Please consider this change that would have a massive cumulated impact on all the people using this template, both in time for plugin developers and in computation for our friends @ GitHub
Alternatives you've considered
Waiting or going back and forth on GitHub without being focused long enough on another task or even canceling some workflows
Additional context
No response
The text was updated successfully, but these errors were encountered:
Thanks for your detailed suggestions, we are very much interested in optimizing the build/CI performance for plugin projects.
The suggested optimizations to skip execution of Gradle tasks make a number of assumptions that may or may not be true in certain plugin projects (e.g., fixed folder names, assuming source directory folder names, etc.). While it certainly can be done and documented, it seems more suitable to try and optimize the GH actions workflow itself without relying on any "hardcoded" rules that all projects started from this template must obey to work.
We'll look into the other points and welcome any further suggestions.
Describe the need of your request
Currently, the CI workflow is extremely slow (10-20 min average). It's not a big deal, but when you have to accumulate a push, a dependency update merge, another run for another dependency PR, another one for the second merge, and so on, it becomes very time-consuming and blocking for no real reason.
I say blocking because you have to sequentially wait for each one so that your release draft can actually be created, so you can finally publish it, wait for another PR workflow with the CHANGELOG.md update, and a final last one when you merge the changelog PR.
It's tedious, slow, and uses GH Actions bandwidth for practically nothing. I can sometimes waste 30-60min for a 5-line fix that I release alongside 2 Renovate PRs.
Proposed solution
The solution lies in 2 words: conditional execution.
I've used this technique in some of my recent projects, where I make my CI smart: it only runs the steps it needs to run depending on the changes I make.
What would I change for our case? 2 things:
paths
/paths-ignore
options to run or not run jobs at workflow-levelstep
-levelHow would I apply them?
build
,test
,inspectCode
, andverify
when no change is made to any gradle dependency/src codetest
if there is notest
folderreleaseDraft
if a release with the same name has already been releasedtest
,inspectCode
andverify
if the change is only related to gradle dependenciesWhat would those changes mean?
Taking our same example above, we'd go from:
To:
Would that be a big deal?
With rough average estimations for a relatively medium codebase of:
build
: 2 mintest
: 2 min (even without any test)inspectCode
: 6 minverify
: 7 minAnd being generous because
Post Setup Gradle
can be extremely slow, sometimes doubling the job's duration but that's not on you, we would go from ~72 min (8 * 9 min) to ~19 min (1 * 9 min + 5 * 2 min + 2 * 0s) of run for a casual workflow with hotfix releases and dependency management.Additional actions improvements
If you want to go beyond, maybe improving the run duration of your own actions, especially
./gradlew runPluginVerifier
andJetBrains/qodana-action
, which are the slowest of all (maybe parallelizing stuff like IDE checks), it'd be amazing!About the Release workflow, it would also be great not to have it fail if the CHANGELOG.md is already up-to-date!
If you guys also have an idea of why and how
Post Setup Gradle
can take up to 4-5 min, I'd love to hear from you!Additionally, I'd love to see Qodana reports in PRs not getting sent on closed/merged PRs.
Conclusion
Please consider this change that would have a massive cumulated impact on all the people using this template, both in time for plugin developers and in computation for our friends @ GitHub
Alternatives you've considered
Waiting or going back and forth on GitHub without being focused long enough on another task or even canceling some workflows
Additional context
No response
The text was updated successfully, but these errors were encountered: