Testing tips
27 May 2019I’ve done a fair amount of data heavy testing recently and here’s some tricks that make things a little easier.
Fixtures
For this example let’s imagine we have a website selling content subscriptions. We have 3 levels of membership:
enum Membership: Int, Comparable {
case free
case basic
case pro
}
func < (lhs: Membership, rhs: Membership) -> Bool {
return lhs.rawValue < rhs.rawValue
}Our User type contains a membership value and a function to determine if a user can access some content:
struct User {
let membership: Membership
func canView(content: Content) -> Bool {
return content.eligibleMembership <= membership
}
}Our content is then marked with an eligibleMembership level:
struct Content {
let eligibleMembership: Membership
}To exhaustively test this function we need to write 9 test examples. This is calculated by looking at the inputs to this function.
NB because this is a member function we also count the member variable membership.
User.membership - has 3 cases (free|basic|pro)
Content.eligibleMembership - has 3 cases (free|basic|pro)
3 cases x 3 cases = 9 test examples
Writing 9 separate tests is going to have a lot of duplication and quite frankly I struggle to name 1 test let alone 9 similar tests that vary only by input.
This is where I would use fixtures and some funky formatting to help readability.
func testCanViewContent() {
let fixtures: [(Bool, Membership, Membership)] = [
// expected | userMembership | eligibleMembership
(true, .free, .free),
(true, .basic, .free),
(true, .pro, .free),
(false, .free, .basic),
(true, .basic, .basic),
(true, .pro, .basic),
(false, .free, .pro),
(false, .basic, .pro),
(true, .pro, .pro),
]
for (expected, userMembership, eligibleMembership) in fixtures {
let result = User(membership: userMembership).canView(content: .init(eligibleMembership: eligibleMembership))
XCTAssertEqual(expected, result)
}
}The basic idea here is to loop over an array of tuples that contain the inputs for all of our examples.
In the body of the loop we do the usual arrange/act/assert dance for unit testing.
What I like about the above is:
- It’s compact and all cases can be in one test.
- Being in a tabular layout makes it easier to read.
- I only have to name the test function once.
What’s bad:
- Performing assertions in a loop makes it hard to see which case is failing.
- Xcode will just highlight the
XCTAssert*line, which is not going to help when we run multiple examples.
- Xcode will just highlight the
- It takes a little practice and getting used to.
I think we can do better and solve the first point…
Use #line
To get better feed back #line is your friend.
It adds a little more boilerplate to each fixture but my goal for testing is to make it super easy to read the tests and figure out when things go wrong.
This is why I’m happy to use an odd tabular format and add a sprinkling of boilerplate.
Here’s the above example with #line added to show how it looks:
func testCanViewContent() {
let fixtures: [(Bool, Membership, Membership, UInt)] = [
// expected | userMembership | eligibleMembership | line
(true, .free, .free, #line),
(true, .basic, .free, #line),
(true, .pro, .free, #line),
(false, .free, .basic, #line),
(true, .basic, .basic, #line),
(true, .pro, .basic, #line),
(false, .free, .pro, #line),
(false, .basic, .pro, #line),
(true, .pro, .pro, #line),
]
for (expected, userMembership, eligibleMembership, line) in fixtures {
let result = User(membership: userMembership).canView(content: .init(eligibleMembership: eligibleMembership))
XCTAssertEqual(expected, result, line: line)
}
}The key changes are:
- Update every case to take the
#line. - Update the call to
XCTAssert*to take#line.
With these changes in place Xcode will now highlight the actual fixture that is failing.
Write Comments
Looping back to an earlier point - a goal when writing good tests is that they be easy to read and simple to debug. That’s why it’s really important that you add comments when it makes sense. I often find myself falling into the trap of trying to contort a test function name to reveal all intent but it often ends up in an unreadable mess.
A good comment for a test would explain why some set up is being done.
I also find that it’s sometimes really useful to add // Given, // When and // Then comments throughout a test to help delineate the different phases of the function.
Applying these suggestions to the above example might result in this change:
/// Membership levels are ordered `free < basic < pro`.
/// These tests run all permutations to ensure that a user can only
/// view content that is less than or equal to their membership level.
func testCanViewContent() {
let fixtures: [(Bool, Membership, Membership, UInt)] = [
// expected | userMembership | eligibleMembership | line
(true, .free, .free, #line),
(true, .basic, .free, #line),
(true, .pro, .free, #line),
(false, .free, .basic, #line),
(true, .basic, .basic, #line),
(true, .pro, .basic, #line),
(false, .free, .pro, #line),
(false, .basic, .pro, #line),
(true, .pro, .pro, #line),
]
for (expected, userMembership, eligibleMembership, line) in fixtures {
// Given
let user = User(membership: userMembership)
let content = Content(eligibleMembership: eligibleMembership)
// When
let result = user.canView(content: content)
// Then
XCTAssertEqual(expected, result, line: line)
}
}Add .make() Extensions
If you are writing a lot of tests and you have types that are awkward to create then you’d benefit from creating factory helpers.
Let’s take the following type:
struct User {
let id: String
let username: String
let membership: Membership
}It only has a couple of properties but it will already be a pain to create lots of users especially if for example we was writing tests that only cared about membership.
We can simplify things by creating an extension only within our test target that provides defaults for all values:
extension User {
static func make(id: String = "_", username: String = "_", membership: Membership = .free) -> User {
return .init(id: id, username: username, membership: membership)
}
}With this helper in place we can create new users really easily and just provide the values we care about:
greet(user: User.make(username: "Paul"))
// We can take advantage of type inference and drop the type as well
greet(user: .make(username: "Paul"))Conclusion
Some of these tips are only useful in certain circumstances but I’ve found it really helpful having these techniques in my tool bag.