Testing Boundaries in Swift

17 Feb 2019

Testing is fairly common practice but there are plenty of rough edges that catch people out. One area that causes trouble is testing across module boundaries. In this post I’m going to walk through the evolution of an API and explain how to approach testing at each step, which often requires evolving the API slightly. Understanding how to test different scenarios is important because it empowers us to craft the boundaries we want without having to compromise on testing or aesthetics.

There are a few key approaches that I’m going to cover:

Subclass and Substitute
Use an Interface
Type Erasure

Problem Outline

As always here’s a contrived example - I’ve got two modules:

Main Application

The main application has a PersonRepository that uses a TransientStore (see Storage Module) as its local cache.

final class PersonRepository {
    let cache: TransientStore

    init(cache: TransientStore = .init()) {
        self.cache = cache
    }

    func fetch(id: String) -> Person? {
        return cache.get(key: id).flatMap { try? JSONDecoder().decode(Person.self, from: $0) }
    }

    func store(id: String, person: Person) {
        cache.set(key: id, value: try? JSONEncoder().encode(person))
    }
}

Usage of this repository within the main application would look like:

let repo = PersonRepository()
repo.store(id: "12345", person: .init(name: "Elliot"))
print(repo.fetch(id: "12345")) //=> Person(name: "Elliot")

Storage Module

The Storage module contains a TransientStore which is a type that provides a simple Key/Value store. Here’s the public interface:

public class TransientStore {
    public init()
    public func get(key: String) -> Data?
    public func set(key: String, value: Data?)
}

The relationship between these types is PersonRepository --> TransientStore, which is to say that the PersonRepository has a strong dependency on TransientStore and knows the type by name.

What do we want to test?

Before we dive into analysing the current structure I think it’s important to highlight exactly what I feel is important to test here for this blog post.

From within my main application I want to test the collaboration between PersonRepository and TransientStore - this is the collaboration across the module boundary. In more concrete terms I want to be able to write tests like:

If I call PersonRepository.fetch(id:) it should:
- Invoke TransientStore.get(key:) with the id value that was passed to the original function
- If data is returned it should attempt to JSON decode it
If I call PersonRepository.store(id:person:) it should:
- Attempt to JSON encode the person passed to the original function
- Invoke TransientStore.set(key:value:) with the id from the original function and the encoded person

The above are the high level collaborations, in reality there would be many permutations of these tests to validate what happens for the unhappy paths like invalid input etc.

What I am not interested in for the sake of this blog is testing the behaviour of TransientStore. In a real project I would expect that TransientStore is well tested to ensure that it honours the public contract that it provides.

Subclass and Substitute

With this first iteration I can test this collaboration by subclassing TransientStore and overriding it’s various functions to create a test double. Here’s an implementation of this test double:

class TransientStoreMock: TransientStore {
    var onFetchCalled: (String) -> Data? = { _ in nil }
    var onStoreCalled: (String, Data?) -> Void = { _, _ in }

    override func get(key: String) -> Data? {
        return onFetchCalled(key)
    }

    override func set(key: String, value: Data?) {
        onStoreCalled(key, value)
    }
}

To show how this would be used - here’s the two test cases I mentioned above:

final class PersonRepositoryTests: XCTestCase {
    var transientStoreMock: TransientStoreMock!
    var sut: PersonRepository!

    override func setUp() {
        super.setUp()
        transientStoreMock = TransientStoreMock()
        sut                = PersonRepository(cache: transientStoreMock)
    }

    override func tearDown() {
        transientStoreMock = nil
        sut                = nil
        super.tearDown()
    }

    func testFetch_collaboratesWithTransientStore() {
        let expectedID = "12345"

        transientStoreMock.onFetchCalled = {
            XCTAssertEqual(expectedID, $0)
            return self.encodedPerson
        }

        XCTAssertEqual(.fake(), sut.fetch(id: expectedID))
    }

    func testStore_collaboratesWithTransientStore() {
        let expectedID = "12345"
        var wasCalled = false

        transientStoreMock.onStoreCalled = {
            XCTAssertEqual(expectedID, $0)
            XCTAssertEqual(self.encodedPerson, $1)
            wasCalled = true
        }

        sut.store(id: expectedID, person: .fake())

        XCTAssertTrue(wasCalled)
    }

    var encodedPerson: Data {
        // Happy to use a force cast here because if this fails there is something really wrong.
        return try! JSONEncoder().encode(Person.fake())
    }
}

extension Person {
    static func fake(name: String = "Elliot") -> Person {
        return .init(name: name)
    }
}

This works but there are a few things I’m not keen on:

To actually make this work I need to update TransientStore to be open so that it can be subclassed externally. This is not a great change to be making just to enable tests. The mere addition of the open access control modifier may suggest to an API user that this type is intended to be subclassed.
This only works for class types so we need a different solution for struct and enum.
There is a burden on me as a test writer to know what to override in our TransientStore subclass. If I don’t override the right stuff then my tests will not be isolated and could be causing all kinds of side effects.

Before moving on… if the above technique fits your needs and you don’t share my concerns then by all means use it - there really is no right and wrong if stuff works for your requirements.

Use an Interface

We can resolve all 3 of the issues above by making PersonRepository depend on an interface that we’ll call Store and then make TransientStore depend on the same interface. This has the effect of inverting the direction of the dependency. Doing this would give us the following (notice how the arrows all point away from the concrete details):

PersonRepository --> Store (protocol) <-- TransientStore

Let’t take a look at the changes required to get this working. We’ll update the Storage module first:

+ public protocol Store {
+     func get(key: String) -> Data?
+     func set(key: String, value: Data?)
+ }

- public class TransientStore {
+ public class TransientStore: Store {
      public init()
      public func get(key: String) -> Data?
      public func set(key: String, value: Data?)
  }

Above I’ve added Store as a protocol. TransientStore is almost identical to our first implementation except we are able to remove the open modifier and we conform to Store.

With this change in place we can update the PersonRepository to the following:

  final class PersonRepository {
-     let cache: TransientStore
+     let cache: Store

-     init(cache: TransientStore = .init()) {
+     init(cache: Store = TransientStore()) {
          self.cache = cache
      }

      func fetch(id: String) -> Person? {
          return cache.get(key: id).flatMap { try? JSONDecoder().decode(Person.self, from: $0) }
      }

      func store(id: String, person: Person) {
          cache.set(key: id, value: try? JSONEncoder().encode(person))
      }
  }

The only difference here is that all references to TransientStore have been replaced with Store except for the default argument instantiation in the initialiser.

With this the body of the tests can remain identical but we need to update the test double to conform to a protocol rather than subclassing:

- class TransientStoreMock: TransientStore {
+ class StoreMock: Store {
      var onFetchCalled: (String) -> Data? = { _ in nil }
      var onStoreCalled: (String, Data?) -> Void = { _, _ in }

      func get(key: String) -> Data? {
          return onFetchCalled(key)
      }

      func set(key: String, value: Data?) {
          onStoreCalled(key, value)
      }
  }

As promised this resolves all 3 issues mentioned above and it didn’t really require many changes. The first two are resolved because we have removed the inheritance aspect. The third issue is resolved because if we modify the protocol to add a new requirement then our tests will no longer compile. This gets me to my happy place where I am doing compiler driven development, which means I just fix all the things the compiler complains about.

I do have a gripe with the above solution and it’s that we’ve hidden some details because we are using a protocol but I still had to reference the TransientStore type by name within the PersonRepository, which highlights that TransientStore is still publicly visible. If we look at the public header for our Storage module again we can see that it leaks implementation details:

public protocol Store {
    public func get(key: String) -> Data?
    public func set(key: String, value: Data?)
}

public class TransientStore : Store {
    public init()
    public func get(key: String) -> Data?
    public func set(key: String, value: Data?)
}

As a consumer of the module I might assume that it would be sensible to use TransientStore directly as it’s freely provided in the public API.

Hiding Details

We can resolve the above issue by hiding the concrete TransientStore type entirely. The way to do this is to provide a factory function that will create a TransientStore but it won’t externally reference the TransientStore type. We can then set everything on TransientStore to have internal visibility:

+ public func makeTransientStore() -> Store {
+     return TransientStore()
+ }

- public class TransientStore: Store {
-     public init()
-     public func get(key: String) -> Data?
-     public func set(key: String, value: Data?)
- }
+ class TransientStore: Store {
+     init()
+     func get(key: String) -> Data?
+     func set(key: String, value: Data?)
+ }

It may not seem like we did anything there apart from change some visibility but the end result is the public interface for the Storage module is now much simpler:

  public protocol Store {
      public func get(key: String) -> Data?
      public func set(key: String, value: Data?)
  }

+ public func makeTransientStore() -> Store

- public class TransientStore: Store {
-     public init()
-     public func get(key: String) -> Data?
-     public func set(key: String, value: Data?)
- }

As you can see there is no mention of the actual type TransientStore. The function name does include the name but this is just a label it’s not the actual type itself being leaked.

At this point we have a nice seam that allows us to provide alternate Store implementations into our code base, whether that be in tests or in production code.

Type Erasure

Type erasure can be pretty daunting but it’s really useful when you know when it can be utilised. I don’t think I’ll ever get to the point where I use it often enough that I remember how to do it without googling - maybe I’ll end up back on this post in the not too distant future.

Continuing with our example above we might wonder if we can make our API more generic and use any Hashable type as the key. To achieve this in Swift we need to add an associatedtype to the Store protocol and use the new type where we was previously hardcoding the String type:

  public protocol Store {
+     associatedtype Key: Hashable

-     public func get(key: String) -> Data?
-     public func set(key: String, value: Data?)
+     public func get(key: Key) -> Data?
+     public func set(key: Key, value: Data?)
  }

Updating the TransientStore to conform to this interface requires that we make the class generic:

- class TransientStore: Store {
-     func get(key: String) -> Data?
-     func set(key: String, value: Data?)
- }
+ class TransientStore<Key: Hashable>: Store {
+     func get(key: Key) -> Data?
+     func set(key: Key, value: Data?)
+ }

The changes so far are valid but the compiler starts getting very unhappy with our factory function for creating a TransientStore

public func makeTransientStore() -> Store { //=> Protocol 'Store' can only be used as a generic constraint because it has Self or associated type requirements
    return TransientStore() //=> Return expression of type 'TransientStore<_>' does not conform to 'Store'
}

This isn’t going to work because the associatedtype means that we can’t use Store in the following places:

As the return type of this function.
As the type of cache variable in PersonRepository.

We have two options to get around this restriction.

1) Forget about the interface approach and go back to using the concrete type directly - just like in the problem statement.
2) Create a type eraser that acts as a wrapper over our concrete types.

As you can tell from the less than positive wording of 1 I’m not going to go that route in this post. Again if this is the right solution for your code base then go ahead and use it.

The mechanics of what we will do is:

A) Create a concrete type which follows the naming convention of adding Any to the beginning of our type e.g. AnyStore.
B) The AnyStore will be generic over the key’s type where Key: Hashable.
C) When instantiating an AnyStore<Key> you will need to provide an instance to wrap, which will need to conform to Store.
D) Replace references to Store within function return types or variable declarations with our new AnyStore<Key> type.

Let’s start with the type eraser (steps A - C):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public class AnyStore<Key: Hashable>: Store {
    let _get: (Key) -> Data?
    let _set: (Key, Data?) -> Void

    public init<Concrete: Store>(_ store: Concrete) where Concrete.Key == Key {
        _get = store.get(key:)
        _set = store.set(key:value:)
    }

    public func get(key: Key) -> Data? {
        return _get(key)
    }

    public func set(key: Key, value: Data?) {
        _set(key, value)
    }
}

Line 1 is defining our new type and stating that it’s generic over a Key type that must be Hashable.

Lines 5-8 is where most of the heavy lifting is done. We are taking in another concrete type that conforms to Store and grabbing all of it’s functions and placing them into some variables 2-3. By doing this it means that we can implement the Store interface get(key:) and set(key:value:) and then delegate to the functions that we captured.

With this in place we move onto updating any place where Store was mentioned as a return type or a variable’s type and change to use our new type eraser.

- public func makeTransientStore() -> Store {
-     return TransientStore()
- }
+ public func makeTransientStore<Key>() -> AnyStore<Key> {
+     return AnyStore(TransientStore())
+ }

  final class PersonRepository {
-     let cache: Store
+     let cache: AnyStore<String>

-     init(cache: Store = makeTransientStore()) {
+     init(cache: AnyStore<String> = makeTransientStore()) {
          self.cache = cache
      }

      func fetch(id: String) -> Person? {
          return cache.get(key: id).flatMap { try? JSONDecoder().decode(Person.self, from: $0) }
      }

      func store(id: String, person: Person) {
          cache.set(key: id, value: try? JSONEncoder().encode(person))
      }
  }

There were surprisingly few changes required to get this to work.

What did we just do?

Let’s look at the public interface for the Storage module:

  public protocol Store {
-     public func get(key: String) -> Data?
-     public func set(key: String, value: Data?)

+     associatedtype Key : Hashable

+     public func get(key: Key) -> Data?
+     public func set(key: Key, value: Data?)
  }

+ public class AnyStore<Key: Hashable>: Store {
+     public init<Concrete: Store>(_ store: Concrete) where Key == Concrete.Key
+     public func get(key: Key) -> Data?
+     public func set(key: Key, value: Data?)
+ }

- public func makeTransientStore() -> Store
+ public func makeTransientStore<Key: Hashable>() -> AnyStore<Key>

We’ve had to expose a new concrete type AnyStore in order to accommodate the fact that we wanted Store to be generic. Exposing a new concrete type may seem at odds with the idea of relying on abstractions over concretions but I tend to think of this kind of type erasure as a fairly abstract wrapper that exists solely to hide concrete implementations.

Expanding our Type Erasure

To really ground our understanding let’s make our Store abstraction more powerful and make it work for any value that is Codable instead of just working with Data. The current method of working with Data directly pushes complexity onto the clients of our Store API as they have to handle marshalling to and from Data.

First let’s see how this change will actually simplify our API usage:

  final class PersonRepository {
-     let cache: AnyStore<String>
+     let cache: AnyStore<String, Person>

-     init(cache: AnyStore<String> = makeTransientStore()) {
+     init(cache: AnyStore<String, Person> = makeTransientStore()) {
          self.cache = cache
      }

      func fetch(id: String) -> Person? {
-         return cache.get(key: id).flatMap { try? JSONDecoder().decode(Person.self, from: $0) }
+         return cache.get(key: id)
      }

      func store(id: String, person: Person) {
-         cache.set(key: id, value: try? JSONEncoder().encode(person))
+         cache.set(key: id, value: person)
      }
  }

To make the above work here’s the modifications required to add the new generic to the Store protocol and feed it through our AnyStore type eraser:

- public class AnyStore<Key: Hashable>: Store {
-     public init<Concrete: Store>(_ store: Concrete) where Key == Concrete.Key
-     public func get(key: Key) -> Data?
-     public func set(key: Key, value: Data?)
- }
+ public class AnyStore<Key: Hashable, Value: Codable>: Store {
+     public init<Concrete: Store>(_ store: Concrete) where Key == Concrete.Key, Value == Concrete.Value
+     public func get(key: Key) -> Value?
+     public func set(key: Key, value: Value?)
+ }

  public protocol Store {
      associatedtype Key: Hashable
+     associatedtype Value: Codable

-     public func get(key: Key) -> Data?
-     public func set(key: Key, value: Data?)
+     public func get(key: Key) -> Value?
+     public func set(key: Key, value: Value?)
  }

- public func makeTransientStore<Key: Hashable>() -> AnyStore<Key>
+ public func makeTransientStore<Key: Hashable, Value: Codable>() -> AnyStore<Key, Value>

Conclusion

That was a lot to go through and it got pretty difficult at the end. I covered a few different methods for testing that have various tradeoffs but are all useful for helping to test across boundaries to ensure that objects are collaborating correctly.

Hopefully the above will demonstrate some of the techniques that can be used to design clean boundaries without compromising because we couldn’t figure out a way to test things.

paul-samuels.com