On starting GradleRunner with assertions enabled

[tudortimi]

Expected Behavior

Gradle Test Kit is used for testing, which is the best time to activate assertions in production code. It should be possible to easily enable assertions when running builds through GradleRunner. Ideally, assertions should be on by default.

Current Behavior (optional)

When using Gradle Test Kit, assertions aren't enabled by default in GradleRunner invocations. This means that any assert statements that the user might rely on during testing are not enabled. Users have to explicitly activate assertions by passing -Dorg.gradle.jvmargs="-ea" to GradleRunner.withArguments().

Context

I'm ashamed to say that I spent quite some time figuring out why assertions weren't firing. I was confused by the fact that assertions were enabled in my Test tasks. Eventually, it dawned on me that GradleRunner instances run in their own java processes, with their own options.

Currently, the user has to activate assertions by passing -Dorg.gradle.jvmargs="-ea" to GradleRunner.withArguments(), next to other arguments that are actually related to the test being run. This reduces readability of the test, because it's not as focused anymore:

def "some test"():
    def result = newGradleRunner()
            .withArguments( '-Dorg.gradle.jvmargs="-ea", 'someTask')
            .build()

It's also not possible to extract the creation of a GradleRunner into an own method, which would pre-populate an argument for enabling assertions, because calling withArguments() would override whatever general options we would like to have:

def newGradleRunner():
    return GradleRunner.create()
                .withPluginClasspath()
                .withArguments('-Dorg.gradle.jvmargs="-ea"')

def "some test"():
    def result = newGradleRunner()
            .withArguments('someTask')    // overwrites "-ea"
            .build()

It also requires the user to know and to write up some very low level mechanical details about enabling assertions (the property Gradle users and the name of the java option). A first class method, like GradleRunner.withAssertions() would be much more user friendly.

Ideally, assertions would be enabled by default, as they are for Test tasks. Maybe it would also be possible to take the value from the calling process. If assertions are enabled for the Test tasks that's using the GradleRunner, then assertions would be enabled for the process that GradleRunner is started in.

As a workaround, I'm currently using withDebug(true), because I don't need to pass environment variables (yet at least). It's also possible to provide a predefined builder in my tests to ensure that they all run with assertions:

def newGradleRunner():
    return GradleRunner.create()
                .withPluginClasspath()
                .withDebug(true')

[ljacomet]

This feature request is in the backlog of the relevant team and is prioritized by them.

---

While the decision to add a top level method to enable assertions needs to be taken by the team working on this, it would make sense to have methods allowing to addArguments to enhance composability of GradleRunner setup.