New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test filtering does not work with Quick #552
Comments
I can't install betas on my main work partition, but I'm going to setup a partition with macOS and the beta Xcode to test soon. |
Awesome, thanks @istx25! Let me know how it goes. |
Hello, I was wondering if there is any progress here. I tried the filtering with xCode 8 and it doesn't seem to work. |
I still haven't had the chance to get around to this. My Mac was acting up around this and then I forgot. So no progress yet. Sorry! |
I tried it out:
|
Related to this, is there anyway to use the |
I played around with this using
Providing the full test name (or just the ) to the
Something in the |
Hey there! 👋🏼 I'm a new maintainer for this project and I'm trying to get the next release out ASAP and also clean up old issues and old PRs. I'm closing all issues that no longer seem relevant or are very old / stale. This does not mean this issue is necessarily being rejected. If you are still interested in contributing the changes specified in this issue, please comment to request it be reopened. We appreciate you opening this issue and acknowledge the time and effort you put in to contribute! You're awesome. 💯 However, we are all volunteers here with limited capacity working for free. Unfortunately, that means we must close out stale issues and PRs to move the project forward in a reasonable way. We apologize for the inconvenience and appreciate your understanding! 😎 ✌🏼 |
If it adds anything, the issue is not stale or outdated: I confirm that filtering still doesn't work on Swift 5.6 on Linux (I don't know the associated XCTest version) using |
@jessesquires @phatblat I found out what the issue is, it may only happen on Linux, I don't have access to a Mac computer to test on macOS. Basically XCTest takes one argument for the list of tests to execute (which is what SwiftPM The thing is, it's not one argument = one test case, it's one argument for all selected test cases, separated by a comma 🙃 So when you have a Quick test that's called The incriminating code is here: https://github.com/apple/swift-corelibs-xctest/blob/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-06-26-a/Sources/XCTest/Private/ArgumentParser.swift#L69 When removing the split call it fixes the issue for me. I'm not too sure where to go from there, I guess the real fix would be in XCTest to add quotes or whatever, but it's useless if SwiftPM does not also add quotes when using What do you recommend? Should I make an XCTest issue? I guess it's normal that commas are not handled properly since test functions are supposed to come from actual function names, but since XCTest allows making them dynamically (through In case the answer is "commas are not legal in test cases names", should we consider a workaround in Quick as well? |
Bug filed with the open source XCTest: apple/swift-corelibs-xctest#458 I also filed FB12489187 for the closed-source version, with contents:
I'll leave this open until it's fixed. |
Tested with XCode 14.2, Same is true in the XCode UI test navigator. |
I understand that |
@bartekpacia it's both. As noted above, the swiftpm and xcodebuild tooling assumes that test names are comma-separated, which conflicts with the fact that Quick (as of version 7) uses commas to separate example groups in a single test. You can disable this by setting the Additionally, when invoked through the command line, swiftpm and xcodebuild both assume that I expect these to get fixed, as the new swift-testing package also allows arbitrary test names. As swift-testing matures, I expect the issues that the swiftpm and xcodebuild CLIs have with Quick's unencoded test selectors to go away. I also fully expect there to be other issues, not related to unencoded test selectors. The reason I bring that up is because unencoded test selectors are a very new feature of Quick. But, I also haven't done any (recent, at least) research on what exactly the causes of this are. |
Thanks for answering @younata! Actually I was asking becuase I was just wondering if it is possible to make I experimented myself with the following test: import Quick
import XCTest
class QuickExampleUITests: QuickSpec {
override class func spec() {
it("testA") {
NSLog("Executed test A")
XCTAssertTrue(true)
}
it("testB") {
NSLog("Executed test B")
XCTAssertTrue(true)
}
}
} That test can be executed with:
and it works just fine! So the root source of the problem is indeed Quick using |
Someone with Xcode 8 installed should try the new
xcodebuild test
filters with Quick. Does everything work as expected?The text was updated successfully, but these errors were encountered: