I write a lot of custom command line tools that live alongside my projects. The tools vary in complexity and implementation. From simplest to most involved here’s my high level implementation strategy:
A single file containing a shebang #!/usr/bin/swift.
A Swift Package Manager project of type executable.
A Swift Package Manager project of type executable that builds using sources from the main project (I’ve written about this here).
The same as above but special care has been taken to ensure that the tool can be dockerized and run on Linux.
The hardest part with writing custom tools is knowing how to get started, this post will run through creating a single file tool.
Problem Outline
Let’s imagine that we want to grab our most recent app store reviews, get a high level overview of star distribution of the recent reviews and look at any comments that have a rating of 3 stars or below.
Skeleton
Let’s start by making sure we can get an executable Swift file. In your terminal you can do the following:
The first line is equivalent to just creating a file called reviews with the following contents
#!/usr/bin/swiftprint("It works!!")
It’s not the most exciting file but it’s good enough to get us rolling. The next command chmod u+x reviews makes the file executable and finally we execute it with ./reviews.
Now that we have an executable file lets figure out what our data looks like.
Source data
Before we progress with writing the rest of the script we need to figure out how to get the data, I’m going to do this using curl and jq.
This is a useful step because it helps me figure out what the structure of the data is and allows me to experiment with the transformations that I need to apply in my tool.
First let’s checkout the URL that I grabbed from Stack Overflow (for this example I’m just using the Apple Support app’s id for reviews):
This is a really fast way of experimenting with data and as we’ll see later it’s helpful when we come to write the Swift code.
The result of the jq filter above is that the large feed will be reduced down to an array of objects with just the title, rating and comment.
At this point I’m feeling pretty confident that I know what my data will look like so I can go ahead and write this in Swift.
Network Request in swift
We’ll use URLSession to make our request - a first attempt might look like:
2 we need to import Foundation in order to use URLSession and URL.
6 we’ll use the default session as we don’t need anything custom.
7 to start we’ll just print anything to check this works.
8 let’s not forget to resume the task or nothing will happen.
Taking the above we can return to terminal and run ./reviews…. nothing happened.
The issue here is that dataTask is an asynchronous operation and our script will exit immediately without waiting for the completion to be called.
Modifying the code to call dispatchMain() at the end resolves this:
Heading back to terminal and running ./reviews we should get some output like Optional(41678 bytes) but we’ve also introduced a new problem - the programme didn’t terminate. Let’s fix this and then we can crack on with the rest of our tasks:
Lines 7-10 are covering cases where there is a failure at the task level.
Lines 12-15 are covering errors at the http level.
Lines 20-23 are covering cases where there is no data returned.
The happy path is hidden in lines 17-20.
Side note: Depending on the usage of your scripts you may choose to tailor the level of error reporting and decide if things like force unwraps are acceptable. I tend to find it’s worth putting error handling in as I’ll rarely look at this code, so when it goes wrong it will be a pain to debug without some guidance.
Parsing the data
We can look at the jq filter we created earlier to guide us on what we need to build.
In order to decode an Entry we’ll provide a custom implementation of init(from:) - this will allow us to flatten the data e.g. instead of having entry.title.label we end up with just entry.title. We can do this with the following:
With this done we can wire it all up - we’ll go back to the happy path and add:
do{print(tryJSONDecoder().decode(Response.self,from:data))exit(EXIT_SUCCESS)}catch{print("Failed to decode - \(error.localizedDescription)")exit(EXIT_FAILURE)}
That’s the complicated stuff out of the way - the next part is the data manipulation that makes the tool actually useful.
Processing the data
Let’s start by printing a summary of the different star ratings. The high level approach will be to loop over all the reviews and keep a track of how many times each star rating was used. We’ll then return a string that shows the rating number and then an asterisk to represent the number of ratings.
The other task we wanted to do was print all the comments that had a rating of 3 or less. This is the simpler of the two tasks as we just need to filter the entries and then format for printing:
Creating tools is a lot of fun and isn’t as scary as it might seem at first. We’ve done networking, data parsing and some data munging all in one file with not too much effort, which is very rewarding.
The single file approach is probably best for shorter tasks. In the example above it’s already becoming unwieldy and it would be worth considering moving to a Swift Package Manager tool (maybe that’s a future post).
Asynchronous code is hard and it’s very easy to make subtle mistakes. I look at a real life influenced example where we
accidentally made our unit tests synchronous, therefore reducing the tests effectiveness at catching bugs.
Background
It’s worth repeating that asynchronous code is hard and it’s very easy to make subtle mistakes.
The real life issue manifested in considerably more interesting ways than what follows but the example here is just to highlight the root cause of the problem. One more disclaimer - the code that caused the issue is pretty poorly designed and is a hang over from an old Objective-C code base, which means the following code is in no way idiomatic and one could argue that more modern coding styles would never have allowed this bug.
Apologies out of the way…
We have two functions:
populate(random:) - is a function that will take a Random (a reference type class) and destructively set its number value.
process() - is a synchronous function that creates a Random, feeds it to populate(random:) and then returns the result.
(Yes I know this is a terrible example but please stick with me)
The current tests for this set up look something like this:
Everything is working great and we continue working through our backlog. A year later a new requirement comes in that we should use a fancy new random number generating web service. We integrate this new service and add the infrastructure to make it possible to stub it out in our tests. The end result is not too dissimilar. First we start by updating the RandomNumberGenerator to utilise our new service (in this case I’m just going to call the completion after a short wait).
The above provides a default implementation of remoteNext but allows us to override it in our tests because it’s just a var. The next thing to do is update our tests to provide a stub implementation of remoteNext that we control for testing.
NB: Unit tests are not acceptance tests and in real life we should run the automated/manual acceptance tests before moving on.
Where did it all go wrong?
Some people will have been internally screaming whilst reading the above changes. We’ve made the error of not taking a step back from our implementation and sense checking it. The process function is synchronous but the RandomNumberGenerator.populate(random:) function was updated to be asynchronous - this is not going to work. The issue has been completely masked by the fact that our unit tests were accidentally making the asynchronous RandomNumberGenerator.populate(random:) synchronous.
What does it mean to make “the asynchronous RandomNumberGenerator.populate(random:) synchronous”? Let’s demonstrate by changing the tests to be truly async again:
In lines 6-8 I’m executing the completion within a DispatchQueue.async instead of calling it immediately. With this change we now get a failing test XCTAssertEqual failed: ("42") is not equal to ("0") -, which is what we expect.
Now that we have a failing test to guide us we have a few options to resolve this. Let’s update the test first to what we believe the api should now be in a fully asynchronous world:
finalclassStep4:XCTestCase{functestProcess(){letrandomNumberGenerator=RandomNumberGenerator()randomNumberGenerator.remoteNext={completioninDispatchQueue.main.async{completion(42)}}weakvarcompletionCalled=expectation(description:"Completion was called.")process(randomNumberGenerator:randomNumberGenerator){numberinXCTAssertEqual(42,number)completionCalled?.fulfill()}wait(for:[completionCalled!],timeout:0.1)}}
Lines 11, 15 and 18 are added to take advantage of the asynchronous testing features of XCTest.
Line 13 represents how the new API will look when it has been made asynchronous.
Finally we need to update the production code to make this test pass:
The source transformation here boils down to adding completion handlers to RandomNumberGenerator.populate and process and wiring everything up. With this in place we have passing unit tests.
Conclusion
By accidentally making our asynchronous code synchronous in our tests we are not validating our code in a realistic situation. If remoteNext is performing networking then these tests are actually testing an impossible situation as we would never expect an immediate response.
In the example above we changed the implicit contract of our implementation by accident, which lead to incorrect behaviour. It could have been equally possible that we deliberately made our tests synchronous to make them easier to write. Whilst not adding the expectation boilerplate is nicer it also leads to potentially fragile tests or weakened acceptance criteria. A DispatchQueue.async may not model the production code exactly but it’s better than changing the functions implicit contract and will not add much overhead to a test run.
What do you test when it comes to Swift’s Codable? I start with some thoughts on stuff that might be useful to test and then dive into TDD’ing an example of a custom Decodable implementation.
Test Candidates
Deciding what we could/should test is worth spending some good time thinking about. To start I’d consider the following:
1) How are required properties handled?
I expect that my type will decode successfully if all the required properties are present and any missing optional properties should have no effect.
2) Are source values being used?
I expect that if my source had the value Paul for the key name then I would end up with a parsed Swift type that reflects these values.
3) Can the data round trip through Codable?
If I am conforming to Codable and not just Decodable or Encodable individually then I expect my type to be able to go through both decoding/encoding without losing any data.
Off the shelf Codable
If you are just conforming to Codable without providing a custom init(from:) or encode(to:) then you probably don’t need to add any of your own unit tests to validate the types decoding/encoding directly. Looking at
the list above we can feel pretty confident that the compiler is generating code that handles all of these cases. This is a great place to be - if I have the following type:
Then I have great documentation that states that both height and width are required and attempting to decode from a source that is missing either will behave in a well defined way. Due to the fact that the implementations of both encoding/decoding are being generated by the compiler I can also be confident that it will generate the correct code to create instances that contain the values from the source.
Custom Codable
For cases where I am defining my own custom implementation of Codable I need to test all the scenarios above. Let’s take the example of wanting to be able to decode the following types:
First up notice that Rectangle and Square are both using the off the shelf Codable implementations provided by the compiler so it’s only Shape that we need to write tests for.
Let’s test drive this
Looking at the JSON we can see that there are keys for type and attributes that are not in our native type. Both of these keys are required in order to parse a Shape so we need to write tests to cover this (don’t worry I simplify the following monstrosity a bit further on):
These two tests are verifying that both those properties are required.
NB: I’ve not placed the fixtures within the main body of ShapesTests on purpose to make it so the main body is all about the tests.
I’ve used private let outside the scope of the unit test class instead of in an extension ShapesTests because I find it tends to be easier to maintain. If I nest inside an extension I end up more often than not breaking tests if I decide to rename the test class name and don’t remember to update all the extensions.
To get these tests to pass I need to write a little bit of implementation. The simplest thing I can do is ensure that both of the keys exist without using their values and then create a default instance of Shape.rectangle (because I need to set self to something). I can do this by decode‘ing from the keyed container and assigning to _:
This implementation is pure nonsense but it makes the tests pass.
I’m not happy with the current tests as they are pretty hard to read and are not doing a great job of describing my intent.
Refactoring tests
I want to refactor these tests to make it really clear what their intent is. Tests that are difficult to read/understand are often just thrown away at a later date, in an effort to avoid this I’d rather spend a little bit of time making things clean.
1) Extract a helper for the assertion
That assertion is pretty difficult to read - at its core it is just checking that decoding will throw a key missing error. I create a helper by copying the original implementation into a new function and making the external interface simpler, which as you’ll see below tidies up the final callsite.
funcAssertThrowsKeyNotFound<T:Decodable>(_expectedKey:String,decoding:T.Type,fromdata:Data,file:StaticString=#file,line:UInt=#line){XCTAssertThrowsError(tryJSONDecoder().decode(decoding,from:data),file:file,line:line){errorinifcase.keyNotFound(letkey,_)?=erroras?DecodingError{XCTAssertEqual(expectedKey,key.stringValue,"Expected missing key '\(key.stringValue)' to equal '\(expectedKey)'.",file:file,line:line)}else{XCTFail("Expected '.keyNotFound(\(expectedKey))' but got \(error)",file:file,line:line)}}}
The next thing I would consider refactoring is the fixture duplication. For both of the tests above I am essentially taking a working JSON object and stripping away keys one at a time and verifying that the correct error is thrown. I can leverage some good old fashion Key Value Coding to make this simple helper:
Now that the tests are looking cleaner lets add some more to force the next bit of production code to be written. I’m going to chose to just verify that type is utilised correctly. I have a feeling this test will be temporary as a later test will make it redundant but let’s write it to keep things moving:
functestDecoding_whenSquare_returnsASquare()throws{letresult=tryJSONDecoder().decode(Shape.self,from:fixture)ifcase.square=result{XCTAssertTrue(true)}else{XCTFail("Expected to parse a `square` but got \(result)")}}functestDecoding_whenRectangle_returnsARectangle()throws{letresult=tryJSONDecoder().decode(Shape.self,from:fixtureRectangle)ifcase.rectangle=result{XCTAssertTrue(true)}else{XCTFail("Expected to parse a `rectangle` but got \(result)")}}...privateletfixtureRectangle=Data("""
{
"type" : "rectangle",
"attributes" : {
"height" : 200,
"width" : 400
}
}
""".utf8)
These tests are pretty permissive and will allow anything as long as it’s the correct enum case. The simplest thing I can write to make this pass is to hardcode some random shapes:
At this point my tests are now a bit rubbish. I’ve had to provide a new fixtureRectangle which isn’t actually representing a valid source anymore. It makes sense to remove these tests and write some more reasonable assertions.
I’ll start by addressing squares first:
The tests pass and I didn’t change the current implementation of init(from:), which is slightly worrying as I know I hardcoded values. The best thing to do is to write another assertion with different data:
We’ve now written tests that cover the requirements at the top (for the Decodable half of the Codable story). We’ve verified that required keys are in fact required and checked that when you do use a decoder your newly created types take the values from the source.
Looking at this new code (listed below) I can see that adding all these new tests has gotten really ugly:
My concerns here are primarily based around adding lots of new fixtures that are poorly named. I could improve the naming but then I feel like the data definition and usage is really far apart. I’d most likely still have to go hunting to look at the fixtures regardless of the name.
I can start to tidy this up by inlining square, square2, rectangle and rectangle2 and deleting the constants:
A further enhancement would be to bring the fixture data into the body of the test as well. We could use a similar idea from earlier where we just munge some existing data inside the test body where it is used so everything is kept local to the test. Here’s the helper function, which again leverages the power of Key Value Coding:
Using the new helper gives the following tests. The code isn’t as short anymore but all the data is local to the tests making it easier for future readers to figure out what data is changing to drive the various conditions:
Testing Codable implementations isn’t particularly hard but the boilerplate code required can get out of hand pretty quickly. I thought I’d run through a TDD process to get to the final solution as I find this stuff personally interesting and hopefully someone else might to. Hopefully I’ve highlighted some basic stuff to test when looking at custom Decodable implementations and shown that it’s useful to refactor not only the production code but the test code as well.