paul-samuels.com

Thoughts on software development languages, tools and techniques.

@[email protected]

© Paul Samuels 2024. All rights reserved.

Writing custom tools with Swift

12 Jan 2019

I write a lot of custom command line tools that live alongside my projects. The tools vary in complexity and implementation. From simplest to most involved here’s my high level implementation strategy:

The hardest part with writing custom tools is knowing how to get started, this post will run through creating a single file tool.

Problem Outline

Let’s imagine that we want to grab our most recent app store reviews, get a high level overview of star distribution of the recent reviews and look at any comments that have a rating of 3 stars or below.

Skeleton

Let’s start by making sure we can get an executable Swift file. In your terminal you can do the following:

echo '#!/usr/bin/swift\nprint("It works!!")' > reviews
chmod u+x reviews
./reviews

The result will be It works!!

The first line is equivalent to just creating a file called reviews with the following contents

#!/usr/bin/swift
print("It works!!")

It’s not the most exciting file but it’s good enough to get us rolling. The next command chmod u+x reviews makes the file executable and finally we execute it with ./reviews.

Now that we have an executable file lets figure out what our data looks like.

Source data

Before we progress with writing the rest of the script we need to figure out how to get the data, I’m going to do this using curl and jq. This is a useful step because it helps me figure out what the structure of the data is and allows me to experiment with the transformations that I need to apply in my tool.

First let’s checkout the URL that I grabbed from Stack Overflow (for this example I’m just using the Apple Support app’s id for reviews):

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json"

To see how this looks I can pretty print it by piping it through jq:

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json" \
    | jq .
Response structure 
```
{
  "feed": {
    "author": {
      "name": {
        "label": "..."
      },
      "uri": {
        "label": "..."
      }
    },
    "entry": [
      {
        "author": {
          "uri": {
            "label": "..."
          },
          "name": {
            "label": "..."
          },
          "label": ""
        },
        "im:version": {
          "label": "..."
        },
        "im:rating": {
          "label": "..."
        },
        "id": {
          "label": "..."
        },
        "title": {
          "label": "..."
        },
        "content": {
          "label": "...",
          "attributes": {
            "type": "text"
          }
        },
        "link": {
          "attributes": {
            "rel": "related",
            "href": "..."
          }
        },
        "im:voteSum": {
          "label": "..."
        },
        "im:contentType": {
          "attributes": {
            "term": "Application",
            "label": "Application"
          }
        },
        "im:voteCount": {
          "label": "..."
        }
      }
    ],
    "updated": {
      "label": "..."
    },
    "rights": {
      "label": "..."
    },
    "title": {
      "label": "..."
    },
    "icon": {
      "label": "..."
    },
    "link": [
      {
        "attributes": {
          "rel": "...",
          "type": "text/html",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "self",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "first",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "last",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "previous",
          "href": "..."
        }
      },
      {
        "attributes": {
          "rel": "next",
          "href": "..."
        }
      }
    ],
    "id": {
      "label": "..."
    }
  }
}
```

Looking at the structure I can see that the data I really care about is under feed.entry so I update my jq filter to scope the data a little better:

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json" \
    | jq '.feed.entry'
Response structure 
```
[
  {
    "author": {
      "uri": {
        "label": "..."
      },
      "name": {
        "label": "..."
      },
      "label": ""
    },
    "im:version": {
      "label": "..."
    },
    "im:rating": {
      "label": "..."
    },
    "id": {
      "label": "..."
    },
    "title": {
      "label": "..."
    },
    "content": {
      "label": "...",
      "attributes": {
        "type": "text"
      }
    },
    "link": {
      "attributes": {
        "rel": "related",
        "href": "..."
      }
    },
    "im:voteSum": {
      "label": "..."
    },
    "im:contentType": {
      "attributes": {
        "term": "Application",
        "label": "Application"
      }
    },
    "im:voteCount": {
      "label": "..."
    }
  }
]
```

Finally I pull out the fields that I feel will be important for the tool we are writing:

curl "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json" \
    | jq '.feed.entry[] | {title: .title.label, rating: ."im:rating".label, comment: .content.label}'
Response structure 
```
[
  {
    "title" : "...",
    "rating" : "...",
    "comment" : "..."
  }
]
```

This is a really fast way of experimenting with data and as we’ll see later it’s helpful when we come to write the Swift code. The result of the jq filter above is that the large feed will be reduced down to an array of objects with just the title, rating and comment.

At this point I’m feeling pretty confident that I know what my data will look like so I can go ahead and write this in Swift.

Network Request in swift

We’ll use URLSession to make our request - a first attempt might look like:

1
2
3
4
5
6
7
8

#!/usr/bin/swift
import Foundation

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    print(response as Any)
}).resume()

Taking the above we can return to terminal and run ./reviews…. nothing happened.

The issue here is that dataTask is an asynchronous operation and our script will exit immediately without waiting for the completion to be called. Modifying the code to call dispatchMain() at the end resolves this:

#!/usr/bin/swift
import Foundation

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    print(response as Any)
}).resume()

dispatchMain()

Heading back to terminal and running ./reviews we should get some output like Optional(41678 bytes) but we’ve also introduced a new problem - the programme didn’t terminate. Let’s fix this and then we can crack on with the rest of our tasks:

1
2
3
4
5
6
7
8
9
10
11

#!/usr/bin/swift
import Foundation

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    print(response as Any)
    exit(EXIT_SUCCESS)
}).resume()

dispatchMain()

On line 8 I’ve added an exit, we’ll provide different exit codes later on depending on whether the tool succeeded or not.

To prepare for the next steps we’ll just add some error handling:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#!/usr/bin/swift
import Foundation

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    if let error = error {
        print(error.localizedDescription)
        exit(EXIT_FAILURE)
    }

    guard let httpResponse = response as? HTTPURLResponse, 200..<300 ~= httpResponse.statusCode else {
        print("Invalid response \(String(describing: response))")
        exit(EXIT_FAILURE)
    }

    if let data = data, data.count > 0 {
        print(data as Any)
        exit(EXIT_SUCCESS)
    } else {
        print("No data!!")
        exit(EXIT_FAILURE)
    }
}).resume()

dispatchMain()

Side note: Depending on the usage of your scripts you may choose to tailor the level of error reporting and decide if things like force unwraps are acceptable. I tend to find it’s worth putting error handling in as I’ll rarely look at this code, so when it goes wrong it will be a pain to debug without some guidance.

Parsing the data

We can look at the jq filter we created earlier to guide us on what we need to build.

jq .feed.entry[] | {title: .title.label, rating: ."im:rating".label, comment: .content.label}

We need to dive into the JSON through feed and entry - we can do this by mirroring this structure and using Swift’s Decodable:

struct Response: Decodable {
    let feed: Feed

    struct Feed: Decodable {
        let entry: [Entry]

        struct Entry: Decodable {

        }
    }
}

In order to decode an Entry we’ll provide a custom implementation of init(from:) - this will allow us to flatten the data e.g. instead of having entry.title.label we end up with just entry.title. We can do this with the following:

struct Entry: Decodable {
    let comment: String
    let rating: Int
    let title: String


    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)

        comment = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .comment).decode(String.self, forKey: .label)
        rating  = Int(try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .rating).decode(String.self, forKey: .label))!
        title   = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .title).decode(String.self, forKey: .label)
    }

    private enum CodingKeys: String, CodingKey {
        case comment = "content"
        case rating  = "im:rating"
        case title

        case label
    }
}

With this done we can wire it all up - we’ll go back to the happy path and add:

do {
    print(try JSONDecoder().decode(Response.self, from: data))
    exit(EXIT_SUCCESS)
} catch {
    print("Failed to decode - \(error.localizedDescription)")
    exit(EXIT_FAILURE)
}

That’s the complicated stuff out of the way - the next part is the data manipulation that makes the tool actually useful.

Processing the data

Let’s start by printing a summary of the different star ratings. The high level approach will be to loop over all the reviews and keep a track of how many times each star rating was used. We’ll then return a string that shows the rating number and then an asterisk to represent the number of ratings.

func ratings(entries: [Response.Feed.Entry]) -> String {
    let countedSet = NSCountedSet()

    entries.forEach { countedSet.add($0.rating) }

    return (countedSet.allObjects as! [Int])
        .sorted(by: >)
        .reduce(into: "") { result, key in
            result.append("\(key): \(String(repeating: "*", count: countedSet.count(for: key)))\n")
    }
}

This will yield output like:

5: *****************
4: **
3: *
2: ****
1: **************************

The other task we wanted to do was print all the comments that had a rating of 3 or less. This is the simpler of the two tasks as we just need to filter the entries and then format for printing:

func reviews(entries: [Response.Feed.Entry]) -> String {
    return entries
        .filter { $0.rating <= 3 }
        .map({ """
            (\($0.rating)) - \($0.title)
            > \($0.comment)
            """
        }).joined(separator: "\n\n-\n\n")
}

This will yield output like:

(3) - Love it
> This is my favourite app.

Putting it all together we end up with:

#!/usr/bin/swift
import Foundation

struct Response: Decodable {
    let feed: Feed

    struct Feed: Decodable {
        let entry: [Entry]

        struct Entry: Decodable {
            let comment: String
            let rating: Int
            let title: String


            init(from decoder: Decoder) throws {
                let container = try decoder.container(keyedBy: CodingKeys.self)

                comment = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .comment).decode(String.self, forKey: .label)
                rating  = Int(try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .rating).decode(String.self, forKey: .label))!
                title   = try container.nestedContainer(keyedBy: CodingKeys.self, forKey: .title).decode(String.self, forKey: .label)
            }

            private enum CodingKeys: String, CodingKey {
                case comment = "content"
                case rating  = "im:rating"
                case title

                case label
            }
        }
    }
}

func ratings(entries: [Response.Feed.Entry]) -> String {
    let countedSet = NSCountedSet()

    entries.forEach { countedSet.add($0.rating) }

    return (countedSet.allObjects as! [Int])
        .sorted(by: >)
        .reduce(into: "") { result, key in
            result.append("\(key): \(String(repeating: "*", count: countedSet.count(for: key)))\n")
    }
}

func reviews(entries: [Response.Feed.Entry]) -> String {
    return entries
        .filter { $0.rating <= 3 }
        .map({ """
            (\($0.rating)) - \($0.title)
            > \($0.comment)
            """
        }).joined(separator: "\n\n-\n\n")
}

let url = URL(string: "https://itunes.apple.com/gb/rss/customerreviews/id=1130498044/sortBy=mostRecent/json")!

URLSession.shared.dataTask(with: url, completionHandler: { data, response, error in
    if let error = error {
        print(error.localizedDescription)
        exit(EXIT_FAILURE)
    }

    guard let httpResponse = response as? HTTPURLResponse, 200..<300 ~= httpResponse.statusCode else {
        print("Invalid response \(String(describing: response))")
        exit(EXIT_FAILURE)
    }

    if let data = data, data.count > 0 {
        do {
            let entries = try JSONDecoder().decode(Response.self, from: data).feed.entry
            print(ratings(entries: entries))
            print()
            print(reviews(entries: entries))
            exit(EXIT_SUCCESS)
        } catch {
            print("Failed to decode - \(error.localizedDescription)")
            exit(EXIT_FAILURE)
        }
    } else {
        print("No data!!")
        exit(EXIT_FAILURE)
    }
}).resume()

dispatchMain()

Conclusion

Creating tools is a lot of fun and isn’t as scary as it might seem at first. We’ve done networking, data parsing and some data munging all in one file with not too much effort, which is very rewarding.

The single file approach is probably best for shorter tasks. In the example above it’s already becoming unwieldy and it would be worth considering moving to a Swift Package Manager tool (maybe that’s a future post).

Related Posts