Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only output when tests/builds fail if --quiet flag is present #7216

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

clayellis
Copy link
Contributor

@clayellis clayellis commented Dec 25, 2023

Only output when tests and builds fail if the --quiet flag is present.

Motivation:

Resolves the latest conversation on #4395 concerning how the --quiet flag doesn't do what it advertises when used with swift test and swift build.

Modifications:

  • When not testing in parallel, and --quiet is present, collect output until tests finish and print output to stdout conditionally.
  • Pass LoggingOptions into ParallelTestRunner.
  • Determine if any of the parallel tests failed and, if so, output all test results (regardless of passing status) if --quiet is present.
  • Adds three new test fixtures (TestQuietPass, TestQuietPassFail, TestQuietFail) for testing the quiet flag
  • Adds tests for the --quiet flag when all tests pass, when some tests pass/fail, when all tests fail.
  • Redirect build output to a thread-safe buffered output stream that will be flushed to stdout if a build fails when --quiet is present.
  • Adds tests for the --quiet flag when a build passes and when one fails.

Result:

If tests pass successfully and the --quiet flag is present, nothing will be output. Output as normal when tests fail. The only notable change to output in the failure case is that when running tests in parallel (--parallel), the progress animation is not updated.

If a build succeeds and the --quiet flag is present, nothing will be output. If a build fails with the flag, the build output will appear as normal on stdout.

…ent. Flush quiet buffer to stdout on failures.
@clayellis clayellis marked this pull request as ready for review December 26, 2023 06:08
@clayellis
Copy link
Contributor Author

This is far enough along now that I think it's ready for review. I'd like some feedback on the approaches taken, whether they're in the right direction.

I'd still like to add a few more tests for swift build --quiet and to resolve the discrepancies between a failed build's output with and without the quiet flag. (See this comment.)

@clayellis clayellis changed the title Only output when tests fail if --quiet flag is present Only output when tests/builds fail if --quiet flag is present Dec 26, 2023
@MaxDesiatov
Copy link
Member

@swift-ci test

@MaxDesiatov MaxDesiatov self-assigned this Dec 26, 2023
@MaxDesiatov MaxDesiatov added bug swift test Changes impacting `swift test` tool labels Jan 2, 2024
Copy link
Member

@MaxDesiatov MaxDesiatov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this makes sense, but IMO it should avoid new uses of BufferedOutputByteStream and resolve a few line length formatting nits.

@MaxDesiatov
Copy link
Member

@swift-ci test

@grynspan
Copy link
Contributor

grynspan commented Jan 2, 2024

Does this change take swift-testing into account? If not, are there changes needed in swift-testing to support it?

@MaxDesiatov
Copy link
Member

@swift-ci test

@MaxDesiatov
Copy link
Member

@swift-ci test windows

1 similar comment
@MaxDesiatov
Copy link
Member

@swift-ci test windows

Copy link
Member

@MaxDesiatov MaxDesiatov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The newly added tests need to be fixed on Linux for both swift build and ./Utilities/build-using-self, and on macOS only for the latter.

@MaxDesiatov MaxDesiatov assigned clayellis and unassigned MaxDesiatov Feb 6, 2024
@MaxDesiatov
Copy link
Member

Hi @clayellis, would you be able to resolve conflicts and address review feedback on this PR? Thanks!

@grynspan grynspan requested a review from briancroom April 8, 2024 15:38
@grynspan grynspan self-requested a review April 8, 2024 15:38
@clayellis
Copy link
Contributor Author

Yes, I'll aim to complete this PR by the end of the week (hopefully earlier.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug swift test Changes impacting `swift test` tool
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants