Using pending causes custom Configuration hooks not to run

[JeffBNimble]

If you create a custom Configuration class and add your own custom closures for:

beforeSuite,
afterSuite,
beforeEach
afterEach
and then use pending (instead of it/xit/fit) on your Quick tests, none of the configuration closures for beforeEach and afterEach are called, beforeSuite or afterSuite are called.

I typically begin my testing by writing out my describe/context(s) and use pending on everything. When I run a test with ALL pending and no it/xit/fit, none of the hooks are invoked.

Here is a very simple example of what I'm talking about:

class SQLQueryOperationPendingSpec : QuickSpec {
    override func spec() {
        describe("With a SQLQueryOperation") {

            context("when executing a query without a table") {

                pending("it will throw a MissingTableName exception") {}

            }

            context("when executing a query with named parameters") {

                pending("it will execute the query against the database") {}

            }

            context("when executing a query with an array of parameters") {

                pending("it will execute the query against the database") {}
            }

        }
    }
}

FWIW, I have not tested what happens when I have tests with a mix of pending/it, so I don't know what happens there.

[JeffBNimble]

It screwed my code formatting up for some reason.

[briancroom]

To Quick, a pending test isn't actually a test at all, just a placeholder for one. beforeEach and afterEach are called for each test, but there aren't any tests in your example, resulting in an empty test suite.

[JeffBNimble]

OK, understood. The only possible confusing thing then is the way Quick prints pending tests. Here is what is printed to the console in my scenario.

Pending: it will throw a MissingTableName exception
Pending: it will execute the query against the database
Pending: it will execute the query against the database

Since it only prints the text of the pending item, it's somewhat confusing. If I define lots of pending tests at first, which I commonly do, and some or many have similar pending descriptions, then it looks like things get duplicated since nothing else is printed.

I realize this is relatively minor. Would it be your suggestion then to use unique pending descriptions so as to provide additional context on exactly what is pending?

[briancroom]

I see, interesting.

I imagine Quick should be constructing the full test name (including description/context strings) when printing the statements about the pending tests. That would clarify things, right?

[JeffBNimble]

I think it would. I was trying to do something similar with the hooks, but since they don't run, I'm unable to construct a meaningful description of pending tests.

I was attempting to do something like this:

With a SQLQueryOperation
    when executing a query without a table
        Pending: it will throw a MissingTableName exception
    when executing a query with named parameters
        Pending: it will execute the query against the database
    when executing a query with an array of parameters
        Pending: it will execute the query against the database

[JeffBNimble]

Well I can't seem to get my formatting correct using Markdown. Each line is supposed to be indented properly. But, hopefully you get the idea.

[modocache]

I think pending examples could be vastly improved.

• Printing the full name, including any describe/context descriptions, would certainly be desirable. Perhaps we should change the name of this issue to reflect that this is being discussed here?
• I would also prefer if pending examples weren't printed at the very beginning, but instead printed at the same location as where they would normally be run.
• Pending examples could also emit warnings, in order to ensure developers don't mistakenly assume they are being executed. (The same should be true of focused examples.)
• Finally, RSpec treats pending examples as "expected failures". It runs the examples and, if they pass, emits a warning/test failure in order to notify the developer that they should be reverted to normal examples, not pending ones. It would be great to do this in Quick as well, but we must be careful not to crash the test suite if the example triggers an exception.

Would the following work for pending examples?

Pending: "when executing a query without a table, it will throw a MissingTableName exception"

[JeffBNimble]

I think that's great. I think it's a nice improvement and provides more context.

I also like the idea that pending examples would be expected failures. I'm guessing that means that what I was originally asking for would happen. That is, pending would also have beforeSuite/afterSuite and beforeEach/afterEach run in custom configurations?

The net of what I'm trying to accomplish may come when reporters land, but until then I'm using these custom configuration hooks to generate nice output when the run is complete.

[takecian]

May i work with this issue?
I'd like to work it, to understand Quick.

[modocache]

Of course, @takecian! Please post any questions you have here, if you get stuck trying to figure things out on your own. 👍

• Printing the full name, including any describe/context descriptions, would certainly be desirable.

I think this is a good place to start. You'll need to find the spot where we print() pending examples. If pending examples aren't actually examples or example groups, they need to be--or at least, they need something like the Example.name property. That property builds a name based on the example groups surrounding the example--which is exactly what we want to print here!

[takecian]

I created PR for this issue. At first I started with,

Printing the full name, including any describe/context descriptions, would certainly be desirable.

It would be appreciate for any feedback.

[takecian]

@modocache, if you have spare time, it would be helpful that you review this PR(#491). 👍

[takecian]

@modocache , @briancroom hi, is it possible to review PR(#491)?

[karapigeon]

I will review your PR soon, @takecian.

[karapigeon]

Closing this issue since @takecian has filed a PR which addresses this functionality. Still haven't got around to reviewing it (sorry @takecian!). I'll actually try to get around to it soon. :)

[ikesyo]

This is not resolved yet actually.