v7.0.0 Beta

[younata]

Beta 3 Changes

Quick 7 Beta 3 adds a flag to re-enable the encoded test selector names. This is useful for automated test analysis tools which track tests over time or are broken with the new human-readable test selectors.

Set the QUICK_USE_ENCODED_TEST_SELECTOR_NAMES environment variable and Quick will generate the older-style encoded test selector names (all non alpha-numeric characters will be replaced with underscores).

Thanks to @tikitu and @stonko1994 for their help prioritizing adding this flag.

Beta 2 Changes

Quick 7 Beta 2 changes how test selectors are generated for QuickSpec. Now, both AsyncSpec and QuickSpec will use the unencoded test names as the test selectors. Test selectors are now generated by joining the describe/context blocks leading up to the it block with ", ". This makes test names immensely easier to read. For example, with the following spec:

class MySpec: QuickSpec {
    override class func spec() {
        describe("some feature") {
            context("in one case") {
                it("has a behavior") {}
            }

            context("in another case") {
                it("doesn't have the earlier behavior") {}
            }
        }
    }
}

will generate the following test selectors:

• some feature, in one case, has a behavior
• some feature, in another case, doesn't have the earlier behavior

Beta 1 Changes

Quick 7 changes how Async tests are run. Instead of forcing all tests to run in an async context, Quick 7 provides a separate Spec class for Async Tests. Create an AsyncSpec subclass, and all tests inside of that subclass will run in an async context. Tests inside of QuickSpec subclasses will have a similar behavior to what was in Quick 5.

Additionally, Quick 7 changes how the DSL is defined slightly. In Quick 6 and before, the DSL was defined as a series of global functions, available to be called anywhere. In Quick 7, these functions were moved to be static methods on the new SyncDSLUser (which QuickSpec, Behavior, and QuickConfiguration conform to) and AsyncDSLUser (which AsyncSpec and AsyncBehavior conform to) protocols. This allows us to make sure that you are using the correct DSL for the context, and was necessary for this approach.

For example:

class SynchronousSpec: QuickSpec {
    override class func spec() {
        it("runs synchronously, as you'd expect") {
            var ocean: [String] = []
            DispatchQueue.main.async {
                ocean.append("dolphins")
                ocean.append("whales")
            }
            expect(ocean).toEventually(contain("dolphins", "whales"))
        }
    }
}

class AsynchronousSpec: AsyncSpec {
    override class func spec() {
        it("runs the test in an async context") {
            var ocean: [String] = []
            DispatchQueue.main.async {
                ocean.append("dolphins")
                ocean.append("whales")
            }
            await expect(ocean).toEventually(contain("dolphins", "whales"))
        }
    }
}

We encourage you to check this out, and to share feedback by filing an issue or discussing this in the discussions.

Full Changelog: v6.1.0...v7.0.0-beta.3

---

This discussion was created from the release v7.0.0 Beta 1.

[tikitu]

The new test selectors are really great for readability! But there's one downside: they don't work as well with automated tooling.

For instance the junit output of fastlane scan seems (as far as I can tell) to divide the test selector into "testsuite name" and "testcase classname" and "testcase name" by splitting on the last space, so you get things like

<testsuite name='LegacyAppTests.MDImageCache_spec MDImageCacheKey, uses the field name to create a field-based cache' tests='1' failures='0'>
    <testcase classname='LegacyAppTests.MDImageCache_spec MDImageCacheKey, uses the field name to create a field-based cache' name='key' time='0.016'/>
  </testsuite>

Similarly, it looks like our jenkins test-report plugin splits the testsuite name on the last period character . for grouping: that means if a test string happens to contain a period e.g. PackageName.TheSpecClass A thing reading file.html should do something with it, the jenkins report will group by PackageName.TheSpecClass a thing reading file instead of by PackageName.

I'm not sure what y'all "should" do about this, since arguably it's not your problem but the fault of fastlane, the jenkins plugin, etc. But as a data point: if you offered a configuration option to process the selectors as they used to be (replacing more or less all whitespace and punctuation with underscores), I would turn it on: it's more important for my team to have the automated tooling able to provide meaningful overview, even though that means the individual test names are less informative.

[tikitu]

Oh hey #1211 solves this completely! (I checked out the branch and gave it a spin, works great.)

I wonder whether QUICK_USE_LEGACY_TEST_SELECTOR_NAMES is the right name here, as it seems to imply that this is only supported for backwards-compatibility reasons: perhaps replacing "legacy" with something like "machine parsable" (by contrast with "human readable") or "valid identifier" (which I think is the goal of the replacement) or something in that line would be better.

(Just to check: I'm assuming/hoping you intend to support this format indefinitely for the automated-tooling use case?)

By the way I really like the env-var solution, as it makes it easy to give my team the benefit of the readable names and my automation the machine-parsable format, from the same tests. Thanks!

[younata]

I wonder whether QUICK_USE_LEGACY_TEST_SELECTOR_NAMES is the right name here, as it seems to imply that this is only supported for backwards-compatibility reasons: perhaps replacing "legacy" with something like "machine parsable" (by contrast with "human readable") or "valid identifier" (which I think is the goal of the replacement) or something in that line would be better.

QUICK_USE_ENCODED_TEST_SELECTOR_NAMES is also another option. I went with legacy to refer to the fact that this is older behavior. I specifically did not use "deprecated" because I don't expect to ever remove this.

My desire is to have human-readable test selector names, and this flag is a good option that enables both.

(Just to check: I'm assuming/hoping you intend to support this format indefinitely for the automated-tooling use case?)

Yes, I'll be supporting this format indefinitely. There will always be a mechanism to have Quick generate encoded/machine-parsable test selector names.

[tikitu]

ENCODED is good

[tikitu]

As a further data point, I'm looking to jump from version 5.x to 7.x without stopping at 6, because the always-on-everywhere async needs too much refactoring for our codebase. I'd also very much like to be able to make use of #1166 ! If it's possible to stabilise a major version that doesn't include too many new breaking changes on top of the async stuff, my team would be very appreciative.

[younata]

Closing discussion now that Quick 7 is out.

I want to thank everyone for their help with this!