CustomTestStringConvertible

SwiftTesting’s parameterised tests are really great. I’ve been finding that I often want to give each example a nice name that shows in the test navigator as the default String(describing:) approach doesn’t always hit the mark.

Let’s take the contrived example of testing an enum initialiser that allows the enum to be constructed with a case insensitive string.

Attempt 1

As a first stab we might go for

struct SoftwareTests {
    @Test("Lowercase names can be used")
    func lowercaseNamesCanBeUsed() {
        #expect(Software("macos") == .macOS)
    }

    @Test("Uppercase names can be used")
    func uppercaseNamesCanBeUsed() {
        #expect(Software("MACOS") == .macOS)
    }

    @Test("Unknown names are mapped to unknown")
    func unknownNamesAreMappedToUnknown() {
        #expect(Software("??") == .unknown)
    }
}

This is fine and results in the test navigator showing our tests like this:

└── SoftwareTests
    ├── Lowercase names can be used
    ├── Uppercase names can be used
    └── Unknown names are mapped to unknown

This all looks fairly reasonable but even in this simple example we can see duplication. Each test repeats the exact same pattern in the expectation. Full disclosure I probably wouldn’t bother changing this code as it’s already fairly concise but let’s imagine that the test bodies were a little longer and there was duplicated set up happening in the bodies.

Attempt 2

In this case you’d want to jump to some parameterised tests which might look like this:

struct SoftwareTests {
    @Test(
        "Init is case insensitive and handles unknown cases",
        arguments: [
            (input: "macos", expected: Software.macOS),
            (input: "MACOS", expected: Software.macOS),
            (input: "??", expected: Software.unknown),
        ]
    )
    func initIsCaseInsensitiveAndHandlesUnknownCases(input: String, expected: Software) {
        #expect(Software(input) == expected)
    }
}

The duplication is gone and the different permutations are run in parallel which is great for test performance. The issue is that we’ve made the test navigator view a little less useful as it now looks like this:

└── SoftwareTests
    └── Init is case insensitive and handles unknown cases
        ├── "macos", .macOS
        ├── "MACOS", .macOS
        └── "??", .unknown

Those labels don’t really mean much unless you read the test implementation. Something to note is that the actual arguments declaration in the @Test annotation is using labels to make it easier to read the test set up to know which field is the input vs the expected. Although the code source is enhanced with these labels the test navigator is not so clear.

Attempt 3

Let’s fix the previous issue using the CustomTestStringConvertible protocol

struct SoftwareTests {
    @Test(
        "Init is case insensitive and handles unknown cases",
        arguments: CaseInsensitiveInit.allCases
    )
    func initIsCaseInsensitiveAndHandlesUnknownCases(fixture: CaseInsensitiveInit) {
        #expect(Software(fixture.input) == fixture.expected)
    }

    struct CaseInsensitiveInit: CustomTestStringConvertible, CaseIterable {
        let input: String
        let expected: Software
        let testDescription: String

        static let allCases: [CaseInsensitiveInit] = [
            .init(input: "macos", expected: .macOS, testDescription: "Lowercase names can be used"),
            .init(input: "MACOS", expected: .macOS, testDescription: "Uppercase names can be used"),
            .init(input: "??", expected: .unknown, testDescription: "Unknown names are mapped to unknown"),
        ]
    }
}

With this set up the test navigator is looking much nicer:

└── SoftwareTests
    └── Init is case insensitive and handles unknown cases
        ├── Lowercase names can be used
        ├── Uppercase names can be used
        └── Unknown names are mapped to unknown

We’ve restored the handy naming whilst keeping the ability for the framework to optimise and call all the cases in parallel.

Going further

With the above we have to add boiler plate but the benefits are quite useful. There are common types where we can provide helper functions to make this process a little smoother like booleans. If we create a little helper like this:

struct DescribedBool: CustomTestStringConvertible {
    let testDescription: String
    let value: Bool
}

func boolTestStringConvertible(label: (Bool) -> String) -> [DescribedBool] {
   [
    .init(testDescription: label(false), value: false),
    .init(testDescription: label(true), value: true),
   ]
}

Then we can write tests that can be described a lot easier

@Test("Display an appropriate install state", arguments: boolTestStringConvertible { "installed = \($0)"})
func displayAnAppropriateInstallState(fixture: DescribedBool) {
    // ...
}

NB: The above will hopefully work in future but due to way the macro currently works it doesn’t like there being a closure in the arguments position. We can work around this by adding a little shim

@Test("Display an appropriate install state", arguments: installedCases())
func displayAnAppropriateInstallState(fixture: DescribedBool) {
    // ...
}

private static func installedCases() -> [DescribedBool] {
    boolTestStringConvertible { "installed = \($0)" }
}

With this in place we get nice descriptions like this:

└── SoftwareTests
    └── Display an appropriate install state
        ├── installed = false
        └── installed = true

Conclusion

SwiftTesting parameterised tests are great. It’s also very easy just to slap a load of test cases in simple tuples and exercise a lot of things but maybe lose some clarity in the test navigator around what the tests are doing. Using CustomTestStringConvertible is a nice way to bring some order back and help yourself and other travellers of your code base to navigate some hopefully extensive tests suites.