Speeding up your iOS Swift tests

Are your iOS Swift isolation tests running fast enough? My guess would be “no” unless you have taken deliberate actions to make them run faster. Here’s an approach to reduce the time we spend waiting for our tests to run.


Problem

If I head to Xcode and create a new iOS project with File > Project > Single View App. This gives me a bare bones project which hasn’t yet been sullied with my attempt at coding. I select the option to include unit tests so that everything is wired up and I get a blank test class.

From here I run the tests to get a sense of what my feedback cycle will be:

# Clean and Test
time xcodebuild -scheme SingleViewApp -destination 'platform=iOS Simulator,name=iPad Air 2,OS=latest' -quiet clean test

#=> 1) 38.470
#=> 2) 32.856
#=> 3) 34.413

# Test with warm caches
time xcodebuild -scheme SingleViewApp -destination 'platform=iOS Simulator,name=iPad Air 2,OS=latest' -quiet test

#=> 1) 31.669
#=> 2) 28.552
#=> 3) 28.462

I’m not going to dig into why these run times are soo slow but just know that this is way off the mark. This is a project with no production code and an empty test file but I have to wait ~30 seconds for feedback.


The End Result

Before going through the practical details of the Swift Package Manager based approach to speeding this up let’s look at the end results - this way I can avoid wasting people’s time if they decide the improvements don’t merit reading further.

Here I look at a single module extracted from a production code base. It’s a small module at 4633 lines of Swift, which is made from several smaller modules. It’s executing a modest 110 tests:

Xcode Clean SPM Clean Diff
46.873 22.717 2.06x faster
54.753 19.897 2.75x faster
58.447 19.883 2.93x faster

Clean builds are still painfully slow but they are 2-3x faster when using this technique so that’s still a win.

If you simply rerun the tests without changing any code you get a wild speed up of ~39% but that’s not a realistic test case. So the numbers for cached builds here will take into consideration changing some code to force some recompilation:

Xcode Cached SPM Cached Diff
31.840 7.389 4.30x faster
29.501 7.166 4.11x faster
25.396 6.306 4.02x faster

This is where the numbers get more exciting - ~7 seconds is still an awfully long time to wait for feedback when you are in a flow but it’s much more manageable than 30-60 seconds.


How?

So you made it past the results - let’s dig in. I’ve actually discussed this general idea already in Creating Swift Package Manager tools from your existing codebase (one of my catchier titles) but this is applying it to improving tests.

Prerequisites

  • If your code is well organised in folders then this will be easier
  • The code you want to test can’t import UIKit
    • Ideally you should avoid import UIKit as much as you can as this code will be more portable

Steps

  • In your project directory create a new Swift Package Manager project
  • Symlink your production code into the new project’s Sources folder
  • Symlink your test code into the new project’s Tests folder
  • Update the Package.swift file to reference your new module
  • Run swift test

Worked example of a simple project (not modularised)

Imagine I have a project called MySimpleApp, which has all the business logic separated from the UIKit related stuff. On disk it looks like:

├── MySimpleApp
│   ├── AppDelegate.swift
│   ├── Models                  <- business logic in this folder
│   │   ├── BlogPost.swift
│   │   └── BlogFilter.swift
│   └── ViewController.swift
├── MySimpleApp.xcodeproj
└── MySimpleAppTests
    └── Models
       ├── BlogPostTests.swift
       └── BlogFilterTests.swift

I would start by creating a new SPM project - I’m going to go with the arbitrary name of UIKitless. The swift package init command could be used but it generates a lot of stuff we don’t need so it’s simpler to build it manually here:

mkdir UIKitless
cd UIKitless
mkdir {Sources,Tests}
(cd Sources && ln -s ../../MySimpleApp/Models MySimpleApp)
(cd Tests && ln -s ../../MySimpleAppTests/Models MySimpleAppTests)

Next we need to create a Package.swift to tell SPM how to build all of this. The full file will look like this:

// swift-tools-version:4.0

import PackageDescription

let package = Package(
    name: "tests",
    products: [ ],
    targets: [
        .target(name: "MySimpleApp"),
        .testTarget(name: "MySimpleAppTests", dependencies: [ "MySimpleApp" ]),
    ]
)

With this we can run swift test and everything should work.


Worked example of a more complicated project

If you have modularised your project in any way then it will mean that you are using statements like import SomeOtherModule in your source files. This is not a problem as long as the module you want to import also follows the prerequisites above.

Here’s the additional steps that are required:

  • Symlink in the other module’s sources
  • Update the Package.swift file to tell SPM about this module

The trick is to use the same module name in SPM as is used in your code base e.g. if I import SomeOtherModule in my source files then the target name of my symlink should be SomeOtherModule

(cd Sources && ln -s ../../MySimpleApp/Modules/SomeOtherModule/Sources SomeOtherModule)

The last thing is to update Package.swift to tell it to build this new module and make it a dependency of MySimpleApp:

// swift-tools-version:4.0

import PackageDescription

let package = Package(
    name: "tests",
    products: [ ],
    targets: [
        .target(name: "MySimpleApp", dependencies: [ "SomeOtherModule" ]),
        .target(name: "SomeOtherModule"),
        .testTarget(name: "MySimpleAppTests", dependencies: [ "MySimpleApp" ]),
    ]
)

With this we can run swift test and everything should work.


Conclusion

Whilst this example is only using a small code base and possibly unrealistic tests it shows some promise. Hopefully I’ve shown that depending on how your codebase is structured this could be quite a cheap experiment to run and if it yields good results then you can keep using it. Having done a fair amount of Ruby where I would expect my tests to run in fractions of a second any time I save a file these tests are still mind numbingly slow - these things are improving all the time so I remain optimistic that testing won’t be this painful forever.

Writing custom tools with Swift

I write a lot of custom command line tools that live alongside my projects. The tools vary in complexity and implementation. From simplest to most involved here’s my high level implementation strategy:

  • A single file containing a shebang #!/usr/bin/swift.
  • A Swift Package Manager project of type executable.
  • A Swift Package Manager project of type executable that builds using sources from the main project (I’ve written about this here).
  • The same as above but special care has been taken to ensure that the tool can be dockerized and run on Linux.

The hardest part with writing custom tools is knowing how to get started, this post will run through creating a single file tool.


Problem Outline

Let’s imagine that we want to grab our most recent app store reviews, get a high level overview of star distribution of the recent reviews and look at any comments that have a rating of 3 stars or below.


Skeleton

Let’s start by making sure we can get an executable Swift file. In your terminal you can do the following:

echo '#!/usr/bin/swift\nprint("It works!!")' > reviews
chmod u+x reviews
./reviews

The result will be It works!!

The first line is equivalent to just creating a file called reviews with the following contents

#!/usr/bin/swift
print("It works!!")

It’s not the most exciting file but it’s good enough to get us rolling. The next command chmod u+x reviews makes the file executable and finally we execute it with ./reviews.

Now that we have an executable file lets figure out what our data looks like.


Source data

Before we progress with writing the rest of the script we need to figure out how to get the data, I’m going to do this using curl and jq. This is a useful step because it helps me figure out what the structure of the data is and allows me to experiment with the transformations that I need to apply in my tool.

First let’s checkout the URL that I grabbed from Stack Overflow (for this example I’m just using the Apple Support app’s id for reviews):

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json"

To see how this looks I can pretty print it by piping it through jq:

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json" \
    | jq .
Response structure
```
{
  "feed": {
    "author": {
      "name": {
        "label": "..."
      },
      "uri": {
        "label": "..."
      }
    },
    "entry": [
      {
        "author": {
          "uri": {
            "label": "..."
          },
          "name": {
            "label": "..."
          },
          "label": ""
        },
        "im:version": {
          "label": "..."
        },
        "im:rating": {
          "label": "..."
        },
        "id": {
          "label": "..."
        },
        "title": {
          "label": "..."
        },
        "content": {
          "label": "...",
          "attributes": {
            "type": "text"
          }
        },
        "link": {
          "attributes": {
            "rel": "related",
            "href": "..."
          }
        },
        "im:voteSum": {
          "label": "..."
        },
        "im:contentType": {
          "attributes": {
            "term": "Application",
            "label": "Application"
          }
        },
        "im:voteCount": {
          "label": "..."
        }
      }
    ],
    "updated": {
      "label": "..."
    },
    "rights": {
      "label": "..."
    },
    "title": {
      "label": "..."
    },
    "icon": {
      "label": "..."
    },
    "link": [
      {
        "attributes": {
          "rel": "...",
          "type": "text/html",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "self",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "first",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "last",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "previous",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "next",
          "href": "..."
        }
      }
    ],
    "id": {
      "label": "..."
    }
  }
}
```

Looking at the structure I can see that the data I really care about is under feed.entry so I update my jq filter to scope the data a little better:

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json" \
    | jq '.feed.entry'
Response structure
```
[
  {
    "author": {
      "uri": {
        "label": "..."
      },
      "name": {
        "label": "..."
      },
      "label": ""
    },
    "im:version": {
      "label": "..."
    },
    "im:rating": {
      "label": "..."
    },
    "id": {
      "label": "..."
    },
    "title": {
      "label": "..."
    },
    "content": {
      "label": "...",
      "attributes": {
        "type": "text"
      }
    },
    "link": {
      "attributes": {
        "rel": "related",
        "href": "..."
      }
    },
    "im:voteSum": {
      "label": "..."
    },
    "im:contentType": {
      "attributes": {
        "term": "Application",
        "label": "Application"
      }
    },
    "im:voteCount": {
      "label": "..."
    }
  }
]
```

Finally I pull out the fields that I feel will be important for the tool we are writing:

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json" \
    | jq '.feed.entry[] | {title: .title.label, rating: ."im:rating".label, comment: .content.label}'
Response structure
```
[
  {
    "title" : "...",
    "rating" : "...",
    "comment" : "..."
  }
]
```

This is a really fast way of experimenting with data and as we’ll see later it’s helpful when we come to write the Swift code. The result of the jq filter above is that the large feed will be reduced down to an array of objects with just the title, rating and comment.

At this point I’m feeling pretty confident that I know what my data will look like so I can go ahead and write this in Swift.


Network Request in swift

We’ll use URLSession to make our request - a first attempt might look like:

1 #!/usr/bin/swift
2 import Foundation
3 
4 let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!
5 
6 URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
7     print(response as Any)
8 }).resume()
  • 2 we need to import Foundation in order to use URLSession and URL.
  • 6 we’ll use the default session as we don’t need anything custom.
  • 7 to start we’ll just print anything to check this works.
  • 8 let’s not forget to resume the task or nothing will happen.

Taking the above we can return to terminal and run ./reviews…. nothing happened.

The issue here is that dataTask is an asynchronous operation and our script will exit immediately without waiting for the completion to be called. Modifying the code to call dispatchMain() at the end resolves this:

#!/usr/bin/swift
import Foundation

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    print(response as Any)
}).resume()

dispatchMain()

Heading back to terminal and running ./reviews we should get some output like Optional(41678 bytes) but we’ve also introduced a new problem - the programme didn’t terminate. Let’s fix this and then we can crack on with the rest of our tasks:

 1 #!/usr/bin/swift
 2 import Foundation
 3 
 4 let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!
 5 
 6 URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
 7     print(response as Any)
 8     exit(EXIT_SUCCESS)
 9 }).resume()
10 
11 dispatchMain()

On line 8 I’ve added an exit, we’ll provide different exit codes later on depending on whether the tool succeeded or not.


To prepare for the next steps we’ll just add some error handling:

 1 #!/usr/bin/swift
 2 import Foundation
 3 
 4 let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!
 5 
 6 URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
 7     if let error = error {
 8         print(error.localizedDescription)
 9         exit(EXIT_FAILURE)
10     }
11 
12     guard let httpResponse = response as? HTTPURLResponse, 200..<300 ~= httpResponse.statusCode else {
13         print("Invalid response \(String(describing: response))")
14         exit(EXIT_FAILURE)
15     }
16 
17     if let data = data, data.count > 0 {
18         print(data as Any)
19         exit(EXIT_SUCCESS)
20     } else {
21         print("No data!!")
22         exit(EXIT_FAILURE)
23     }
24 }).resume()
25 
26 dispatchMain()
  • Lines 7-10 are covering cases where there is a failure at the task level.
  • Lines 12-15 are covering errors at the http level.
  • Lines 20-23 are covering cases where there is no data returned.
  • The happy path is hidden in lines 17-20.

Side note: Depending on the usage of your scripts you may choose to tailor the level of error reporting and decide if things like force unwraps are acceptable. I tend to find it’s worth putting error handling in as I’ll rarely look at this code, so when it goes wrong it will be a pain to debug without some guidance.


Parsing the data

We can look at the jq filter we created earlier to guide us on what we need to build.

jq .feed.entry[] | {title: .title.label, rating: ."im:rating".label, comment: .content.label}

We need to dive into the JSON through feed and entry - we can do this by mirroring this structure and using Swift’s Decodable:

struct Response: Decodable {
    let feed: Feed

    struct Feed: Decodable {
        let entry: [Entry]

        struct Entry: Decodable {

        }
    }
}

In order to decode an Entry we’ll provide a custom implementation of init(from:) - this will allow us to flatten the data e.g. instead of having entry.title.label we end up with just entry.title. We can do this with the following:

struct Entry: Decodable {
    let comment: String
    let rating: Int
    let title: String


    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)

        comment = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .comment).decode(String.self, forKey: .label)
        rating  = Int(try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .rating).decode(String.self, forKey: .label))!
        title   = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .title).decode(String.self, forKey: .label)
    }

    private enum CodingKeys: String, CodingKey {
        case comment = "content"
        case rating  = "im:rating"
        case title

        case label
    }
}

With this done we can wire it all up - we’ll go back to the happy path and add:

do {
    print(try JSONDecoder().decode(Response.self, from: data))
    exit(EXIT_SUCCESS)
} catch {
    print("Failed to decode - \(error.localizedDescription)")
    exit(EXIT_FAILURE)
}

That’s the complicated stuff out of the way - the next part is the data manipulation that makes the tool actually useful.


Processing the data

Let’s start by printing a summary of the different star ratings. The high level approach will be to loop over all the reviews and keep a track of how many times each star rating was used. We’ll then return a string that shows the rating number and then an asterisk to represent the number of ratings.

func ratings(entries: [Response.Feed.Entry]) -> String {
    let countedSet = NSCountedSet()

    entries.forEach { countedSet.add($0.rating) }

    return (countedSet.allObjects as! [Int])
        .sorted(by: >)
        .reduce(into: "") { result, key in
            result.append("\(key): \(String(repeating: "*", count: countedSet.count(for: key)))\n")
    }
}

This will yield output like:

5: *****************
4: **
3: *
2: ****
1: **************************

The other task we wanted to do was print all the comments that had a rating of 3 or less. This is the simpler of the two tasks as we just need to filter the entries and then format for printing:

func reviews(entries: [Response.Feed.Entry]) -> String {
    return entries
        .filter { $0.rating <= 3 }
        .map({ """
            (\($0.rating)) - \($0.title)
            > \($0.comment)
            """
        }).joined(separator: "\n\n-\n\n")
}

This will yield output like:

(3) - Love it
> This is my favourite app.

Putting it all together we end up with:

#!/usr/bin/swift
import Foundation

struct Response: Decodable {
    let feed: Feed

    struct Feed: Decodable {
        let entry: [Entry]

        struct Entry: Decodable {
            let comment: String
            let rating: Int
            let title: String


            init(from decoder: Decoder) throws {
                let container = try decoder.container(keyedBy: CodingKeys.self)

                comment = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .comment).decode(String.self, forKey: .label)
                rating  = Int(try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .rating).decode(String.self, forKey: .label))!
                title   = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .title).decode(String.self, forKey: .label)
            }

            private enum CodingKeys: String, CodingKey {
                case comment = "content"
                case rating  = "im:rating"
                case title

                case label
            }
        }
    }
}

func ratings(entries: [Response.Feed.Entry]) -> String {
    let countedSet = NSCountedSet()

    entries.forEach { countedSet.add($0.rating) }

    return (countedSet.allObjects as! [Int])
        .sorted(by: >)
        .reduce(into: "") { result, key in
            result.append("\(key): \(String(repeating: "*", count: countedSet.count(for: key)))\n")
    }
}

func reviews(entries: [Response.Feed.Entry]) -> String {
    return entries
        .filter { $0.rating <= 3 }
        .map({ """
            (\($0.rating)) - \($0.title)
            > \($0.comment)
            """
        }).joined(separator: "\n\n-\n\n")
}

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    if let error = error {
        print(error.localizedDescription)
        exit(EXIT_FAILURE)
    }

    guard let httpResponse = response as? HTTPURLResponse, 200..<300 ~= httpResponse.statusCode else {
        print("Invalid response \(String(describing: response))")
        exit(EXIT_FAILURE)
    }

    if let data = data, data.count > 0 {
        do {
            let entries = try JSONDecoder().decode(Response.self, from: data).feed.entry
            print(ratings(entries: entries))
            print()
            print(reviews(entries: entries))
            exit(EXIT_SUCCESS)
        } catch {
            print("Failed to decode - \(error.localizedDescription)")
            exit(EXIT_FAILURE)
        }
    } else {
        print("No data!!")
        exit(EXIT_FAILURE)
    }
}).resume()

dispatchMain()

Conclusion

Creating tools is a lot of fun and isn’t as scary as it might seem at first. We’ve done networking, data parsing and some data munging all in one file with not too much effort, which is very rewarding.

The single file approach is probably best for shorter tasks. In the example above it’s already becoming unwieldy and it would be worth considering moving to a Swift Package Manager tool (maybe that’s a future post).

Accidentally Synchronous Tests

Asynchronous code is hard and it’s very easy to make subtle mistakes. I look at a real life influenced example where we accidentally made our unit tests synchronous, therefore reducing the tests effectiveness at catching bugs.


Background

It’s worth repeating that asynchronous code is hard and it’s very easy to make subtle mistakes.

The real life issue manifested in considerably more interesting ways than what follows but the example here is just to highlight the root cause of the problem. One more disclaimer - the code that caused the issue is pretty poorly designed and is a hang over from an old Objective-C code base, which means the following code is in no way idiomatic and one could argue that more modern coding styles would never have allowed this bug.

Apologies out of the way…

We have two functions:

  • populate(random:) - is a function that will take a Random (a reference type class) and destructively set its number value.
  • process() - is a synchronous function that creates a Random, feeds it to populate(random:) and then returns the result.

(Yes I know this is a terrible example but please stick with me)

The current tests for this set up look something like this:

class Random {
    var number = Int(0)
}

class RandomNumberGenerator {
    func populate(random: Random) {
        random.number = 42
    }
}

func process(randomNumberGenerator: RandomNumberGenerator = .init()) -> Int {
    let random = Random()
    randomNumberGenerator.populate(random: random)
    return random.number
}

final class Step1: XCTestCase {
    func testProcess() {
        XCTAssertEqual(42, process())
    }
}

Everything is working great and we continue working through our backlog. A year later a new requirement comes in that we should use a fancy new random number generating web service. We integrate this new service and add the infrastructure to make it possible to stub it out in our tests. The end result is not too dissimilar. First we start by updating the RandomNumberGenerator to utilise our new service (in this case I’m just going to call the completion after a short wait).

class RandomNumberGenerator {
    var remoteNext: (@escaping (Int) -> Void) -> Void = { completion in
        DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 3) {
            completion(42)
        }
    }

    func populate(random: Random) {
        remoteNext { number in
            random.number = number
        }
    }
}

The above provides a default implementation of remoteNext but allows us to override it in our tests because it’s just a var. The next thing to do is update our tests to provide a stub implementation of remoteNext that we control for testing.

final class Step2: XCTestCase {
    func testProcess() {
        let randomNumberGenerator = RandomNumberGenerator()

        randomNumberGenerator.remoteNext = { completion in
            completion(42)
        }

        XCTAssertEqual(42, process(randomNumberGenerator: randomNumberGenerator))
    }
}

The test passed so we move on to our next task.

NB: Unit tests are not acceptance tests and in real life we should run the automated/manual acceptance tests before moving on.


Where did it all go wrong?

Some people will have been internally screaming whilst reading the above changes. We’ve made the error of not taking a step back from our implementation and sense checking it. The process function is synchronous but the RandomNumberGenerator.populate(random:) function was updated to be asynchronous - this is not going to work. The issue has been completely masked by the fact that our unit tests were accidentally making the asynchronous RandomNumberGenerator.populate(random:) synchronous.

What does it mean to make “the asynchronous RandomNumberGenerator.populate(random:) synchronous”? Let’s demonstrate by changing the tests to be truly async again:

 1 final class Step3: XCTestCase {
 2     func testProcess() {
 3         let randomNumberGenerator = RandomNumberGenerator()
 4 
 5         randomNumberGenerator.remoteNext = { completion in
 6             DispatchQueue.main.async {
 7                 completion(42)
 8             }
 9         }
10 
11         XCTAssertEqual(42, process(randomNumberGenerator: randomNumberGenerator))
12     }
13 }

In lines 6-8 I’m executing the completion within a DispatchQueue.async instead of calling it immediately. With this change we now get a failing test XCTAssertEqual failed: ("42") is not equal to ("0") -, which is what we expect.

Now that we have a failing test to guide us we have a few options to resolve this. Let’s update the test first to what we believe the api should now be in a fully asynchronous world:

 1 final class Step4: XCTestCase {
 2     func testProcess() {
 3         let randomNumberGenerator = RandomNumberGenerator()
 4 
 5         randomNumberGenerator.remoteNext = { completion in
 6             DispatchQueue.main.async {
 7                 completion(42)
 8             }
 9         }
10 
11         weak var completionCalled = expectation(description: "Completion was called.")
12 
13         process(randomNumberGenerator: randomNumberGenerator) { number in
14             XCTAssertEqual(42, number)
15             completionCalled?.fulfill()
16         }
17 
18         wait(for: [ completionCalled! ], timeout: 0.1)
19     }
20 }
  • Lines 11, 15 and 18 are added to take advantage of the asynchronous testing features of XCTest.
  • Line 13 represents how the new API will look when it has been made asynchronous.

Finally we need to update the production code to make this test pass:

class RandomNumberGenerator {
    var remoteNext: (@escaping (Int) -> Void) -> Void = { completion in
        DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 3) {
            completion(42)
        }
    }

    func populate(random: Random, completion: @escaping (Random) -> Void) {
        remoteNext { number in
            random.number = number
            completion(random)
        }
    }
}

func process(randomNumberGenerator: RandomNumberGenerator = .init(), completion: @escaping (Int) -> Void) {
    let random = Random()
    randomNumberGenerator.populate(random: random) { random in
        completion(random.number)
    }
}

The source transformation here boils down to adding completion handlers to RandomNumberGenerator.populate and process and wiring everything up. With this in place we have passing unit tests.


Conclusion

By accidentally making our asynchronous code synchronous in our tests we are not validating our code in a realistic situation. If remoteNext is performing networking then these tests are actually testing an impossible situation as we would never expect an immediate response.

In the example above we changed the implicit contract of our implementation by accident, which lead to incorrect behaviour. It could have been equally possible that we deliberately made our tests synchronous to make them easier to write. Whilst not adding the expectation boilerplate is nicer it also leads to potentially fragile tests or weakened acceptance criteria. A DispatchQueue.async may not model the production code exactly but it’s better than changing the functions implicit contract and will not add much overhead to a test run.