Meta macros are pretty nifty but trying to follow how they work can really challenge the limits of your mental stack frame.
Today I'm going to try and follow macro expansion from start to finish using libextobjc to do some basic metaprogramming. This is going to be a bumpy ride but as always it's great to see how different techniques can be used to solve problems.
Let's start with a fictitious problem that I would like to solve with some metaprogramming. I would probably never do this in a real project but it gives me a realistic use case to work through.
Imagine I want my view controllers to fail hard if I forget to connect up an outlet in the xib file. I could start with something like this:
whoa that's a lot of repetition and it's not going to scale well. What would be great is if I could write some code that would write this repetitious code for me, ideally I would just type something like this:
This seems a lot DRY'er so let's aim for something similar to this and see how we get on.
Down the rabbit hole we go
After examining the metamacros header I can see that there is a foreach macro that sounds like it would be perfect for this task.
After reading the docs I can see that the MACRO argument should be the name of a macro that takes two arguments in the form of MACRO(INDEX, ARG). The INDEX parameter will be the index of the current iteration in the for loop and the ARG parameter will be the argument for the current iteration in the for loop.
So I need to start of by defining a macro that takes these two arguments and expands to the NSParameterAssert that I want. Here's a first stab at such a macro
I don't actually care to use the value of INDEX so it is ignored. This is the macro that will be used within the metamacro_foreach and will eventually expand into the required NSParameterAsserts.
In each of the following examples I'll show the input (starting macro) above the 3 dashes and what this would theoretically expand into below the 3 dashes. I'll optionally show any macro definitions at the top of the code block.
Here's how my OUTLET_ASSERT macro will work:
Now let's see how we can use metamacro_foreach to write the PASAssertConnections macro that will take in a list of ivar names and expand them to the required NSParameterAsserts.
In this case I pass OUTLET_ASSERT as the macro to use on each iteration. I pass ; to use as a separator between iterations, which will terminate each NSParameterAssert. Then finally a comma separated list of ivar names that we are going to iterate over and generate the NSParameterAsserts for.
The metamacro_argcount macro uses another macro called metamacro_at. The metamacro_at is similar in concept to indexing into an array like myArray[index]. In plain English this macro is the same as "give me the nth item in the following list".
The metamacro_argcount macro uses a clever little trick. If we put the numbers from INDEX down to 0 into an array and then ask for the value at INDEX we would get the last number, which would be 0. If we preprend something to the beginning of this array and asked for the value at INDEX again we would now get 1.
Let's see this in Objective-C so it's easier to picture:
The relationship is that when you prepend an argument to the array you shift all of the numeric values to the right by one step, which moves a higher number into the index that is being fetched. This of course only works up to the value of INDEX - so we can tell that this particular implementation of metamacros only supports 20 arguments.
NB - this implementation of metamacros requires at least one argument to be given when using metamacro_argcount.
You'll see the trick of inserting __VA_ARGS__ into argument lists at different points used a few times so it's worth making sure you understand what is happening above.
Great there's our old friend metamacro_concat so we don't need to look up how that works again to know that this will expand like this:
The change is very subtle. The 20 has moved from being an argument to now actually being part of the macro name. So now we need to look up metamacro_at20
It turns out that there are variants of metamacro_at defined for 0 to 20, which allows you to access any of the first 20 arguments from the __VA_ARGS__ arguments.
This is another common trick you'll see with metamacros, at some point you have to knuckle down and write out multiple versions of the same macro to handle different length argument lists. You'll often see that metamacros are generated by other scripts that allow you to specify how many arguments you would like to support without having to hand roll all the variations of metamacro_at0..N.
To make metamacro_at a little easier to digest I'll examine one of the smaller versions of this macro.
The _0 and _1 arguments are basically used as placeholders to gobble up the items at indices 0 and 1 from the arguments. Then we bundle the rest of the arguments together with .... The newly trimmed __VA_ARGS__ is then passed into metamacro_head
metamacro_head uses the opposite trick to metamacro_at*. In this case we are only interested in the first item and we want to throw away the rest of the __VA_ARGS__ list. This is achieved by grabbing the first argument in FIRST and then collecting the rest with ....
Don't worry the end is now very much in sight, just a couple more painless macro expansions. The previous expansion gives us the new metamacro_foreach_cxt2 macro to check out.
This is another example of macro that has multiple versions defined from 0..20. Each of these foreach macros works by utilising the foreach macro that is defined to take one less argument than itself until we get all the way down to metamacro_foreach_cxt1
We are now at the point where we need to see what MACRO expands to. In this case MACRO is actually the metamacro_foreach_iter macro that we passed in near the beginning and I delayed explaining.
This macro is really just an implementation detail and as such shouldn't be used directly but we still want to see what part it plays:
Nice and simple - metamacro_foreach_iter is just a helper that takes our macro OUTLET_ASSERT and the two arguments that our macro should receive and puts the pieces in the right order to be further expanded into the NSParameterAssert calls that we want.
now I'm actually one step away from achieving this, but if this post has gotten your interest I'll leave that as a simple exercise - it's always better to learn by doing and not just skimming through blog posts hoping to learn by osmosis.
Metaprogramming is normally something that people associate with more dynamic languages like Ruby but there's a whole load of possibilities and cool tricks out there just waiting to be learned. As always I encourage you to join me in peeling back the curtain and seeing that there is normally no magic to be found in your favorite OSS projects.
Follow me as I poke Expecta with a stick to see how it works - it's pretty cool.
It's been a long time since my last post but I thought I'd get stuck into how something that appears simple actually works. That subject will be Expecta matchers. If you don't know what Expecta is then you might want to nip over to the Github repo and skim the Readme then we can get stuck in.
Take the following example:
This line looks so simple but it's hiding a lot of clever techniques that may or may not be useful to keep within your own development bag of tricks.
Let's break this apart and demystify what's really going on.
expect() looks like a function call, so you may assume that there is a function declared somewhere called expect. Thankfully that is not the case or this blog post would be very boring - instead we find this defined as:
At this point you may be wondering why did they bother with this odd chain of macro expansions. The logical reason would most likely be that expect(id actual) is an optional short hand syntax, which is only enabled by defining EXP_SHORTHAND before importing Expecta.h. Without this define you have to use the long hand EXP_expect(id actual) and this is what expands to _EXP_expect with all the additional arguments.
Go ahead and reread that last paragraph a few times if it didn't sink in the first time.
In effect the define for expect saves you from having to type out
every time you want to set up an expectation.
The _EXP_expect function simply creates a new instance of EXPExpect with all the arguments supplied. I'm not going to go over the EXPExpect class as I want to cover the single line of code at the top of this post.
Before we move on though it's worth pointing out that Expecta is really cool as it does not require you to box your arguments. It's the EXPObjectify function that does the work of making sure that if you pass in a primitive like int, float, double, etc then it will box it with an NSValue or NSNumber automatically for you.
to looks simple enough - so why is this interesting? Well knowing that it is used like this to.equal... leads us to the conclusion that it returns an instance of something that responds to equal. Before following the link to look up the definition keep in mind that to is entirely optional and I could validly call expect(2).equal(2); - this should narrow down what to returns.
Yup you may have guessed it to returns an instance of EXPExpect - not just any instance but the instance it was called on - check it out:
This is just a little sprinkling of syntactic sugar. It makes the expectation read better consider expect(2).to.equal(2) vs expect(2).equal(2).
By now you are probably thinking that the interesting stuff is over and equal will just be a property declared on EXPExpect. You may also come to the conclusion that the property will return a pointer to either a block or a function so that it can be invoked with parentheses and an argument equal(2). This is exactly what is happening - kind of...
If you search the EXPExpect class you will not find a property declaration but if you follow the declarationf of equal through you'll land in EXPExpect+equal.h, which looks like this:
This is where we have to make sure our brain is really engaged and step up a gear. Take a breather and join me after the relaxing grey line...
In english this has declared a named category called _equalMatcher on the EXPExpect class. This category declares a single readonly property, which means that in the .m file we would expect to see a single method declared with the signature - (void(^)(id expected))_equal;
NB I showed the mapping of _equal as equal is only used for code completion and there is in fact never an implementation declared for - (void(^)(id expected))equal;
So being inquisitive we jump to the implementation to see how this method is defined and we find more #define magic.
When the two macros are expanded we end up with this (formatting mine):
Yup that's right there are line numbers in this listing as it's a big one.
So let's distill what this category method is actually doing:
If we didn't want to use the macros (not advised at all - only shown for interest sake) then the implementation could remove the added complexity of setting up the DSL and end up with an implementation like this:
This version seems like an awful lot of error prone boiler plate code that a developer would have to write for each matcher. Keep in mind that there are ~25 matchers included with Expecta and you can define your own.
Let's list the required steps for this implementation:
Configuring this instance's properties with blocks for matchBlock (line 7), failureMessageForToBlock (line 18), and failureMessageForNotToBlock (line 22).
Ensures that [self applyMatcher:matcher to:&actual]; is executed at the end of the block that is declared and returned (line 3).
In this implementation there are only 3 steps so this must surely be better?
- In the original list of 5 steps only step 4 was exposed to the developer and the remaining steps were hidden behind macros.
- In this implementation the developer has to know about all 3 steps.
- This means that the developer has 2 extra steps to remember to do and to make matters worse they are wrapping (before + after) steps.
- In the first listing the developer is literally just stating the test requirements, whereas in this version the developer has to know about matchers and how they need to be configured in addition to the test requirements.
So hopefully now I've pulled back the curtain a little the line
won't seem as mysterious.
Well that was a heavy post with a lot to understand and I do apologise if I got any of it wrong - I'm no Expecta expert. Reading code is great fun especially when you get that Eureka moment and you learn something new. The joy of a project like Expecta is that it is all unit tested so you can hack around and change things to test your assumptions - hit test and wait for your theory to be validated with a sea of red or green unit test results.
Storyboards can really speed up development but a problem I run into over and over again is working with identifiers that are set in the storyboard and referenced in code.
Let's take the example of making a segue:
You create a segue and then assign it an identifier in the storyboard... and that's the problem. The identifier is defined in the storyboard, which leaves me with plenty of opportunities to mess up referencing the segue in code.
My normal strategy is to create a constants file that would have a constant with the value of each of these identifiers, which starts to look like the following (forgive the naming):
This works for me and I'm happy, but it seems like a lot of manual graft and can become tiresome to maintain especially in a young project with lots of development spikes. It's also still very possible to make typos and mess up these declarations or for them to become out of sync with the storyboard.
I got to thinking about this problem and two things stood out
I don't like doing things manually and
I don't like that it is so easy to misspell a constant when swapping between storyboard and code (yes I am aware copy and paste exists).
So I started tinkering and I've come up with a rough tool that I've cunningly named sbconstants.
My automation idea is that you add a build script that will extract constants from your storyboards and then dump them into a file.
Grab it with
sudo gem install sbconstants
My first effort has a UI like this (command line app)
% sbconstants -h
Usage: DESTINATION_FILE [options]
-p, --prefix=<prefix> Only match identifiers with <prefix>
-s, --source-dir=<source> Directory containing storyboards
-q, --queries=<queries> YAML file containing queries
-d, --dry-run Output to STDOUT
-v, --verbose Verbose output
To get this working on a project the sbconstantsgem needs to be installed to the system Ruby and then we need to call the executable from a build script. An example call would look like this
NB The argument is the destination file to dump the constants into - this needs to be added manually
Every time this project is built it will parse the storyboard files and pull out any constants and then dump them into PASStoryboardConstants.(h|m). This of course means that PASStoryboardConstants.(h|m) should not be edited by hand as it will just be clobbered any time we build.
The output of running this tool will look something like this:
The constants are grouped by where they were found in the storyboard xml e.g. segue.identifier. This can really help give you some context about where/what/when and why a constant exists.
Options are fun and there are a few to play with - most of these options are really only any good for debugging.
-p, --prefix=<prefix> Only match identifiers with <prefix>
Using the prefix option you can specify that you only want to grab identifiers that start with a certain prefix, which is always nice.
If you don't want to run the tool from the root of your app for some reason you can specify the source directory to start searching for storyboard files. The search is recursive using a glob something like <source-dir>/**/*.storyboard
-d, --dry-run Output to STDOUT
If you just want to run the tool and not write the output to a file then this option will spit the result out to $stdout
-v, --verbose Verbose output
Perhaps you want a little more context about where your identifiers are being grabbed from for debugging purposes. Never fear just use the --verbose switch and get output similar to:
Chances are I've missed some identifiers to search for in the storyboard. You don't want to wait for the gem to be updated or have to fork it and fix it. Using this option you can provide a YAML file that contains a description of what identifers to search for. The current one looks something like this (NB this is a great starting point for creating your own yaml):
This looks a little funky but it's essentially groups of keys and values (both the key and the value can be an array). This actually gets expanded to looks for
I'm not sure if this tool will be of any use to anyone but for the projects I've been tinkering with it seems to work pretty well leaving me confident that my storyboard identifier's will not go out of sync with how they are referenced in code.
If anyone does use this and they like it or they have any issues please report it to me so I can get it fixed up and improve my own workflow.
I realised that it would probably be pretty handy to be able to search all the simulators simultaneously for an app with a certain name.
To do this simply type sidir which shows a list of all the simulators like so
now instead of drilling down by simulator you can just start typing the name of the app
BOOM - simple and it even shows what simulator each is installed on, which was harder than it might seem.
If you want to know more about how I went about implementing this change read on or just go and update your workflow and enjoy.
The longer story
So it happened - as I was commuting home I thought of a slight improvement that I wanted to make to my workflow (the above feature change). Normally this would be no biggie as I can just hack on it and get it working but...
It turns out iOS Dev Weekly liked the workflow and placed it in the weekly newsletter, which means I can't just hack on it anymore for the fear that I could be deeply embarrassed if people download it and find it not working. Whilst I can tolerate tools that break in odd ways when I am the sole user I can imagine other people are not so forgiving.
On the train I took a look at what it would take to implement the change I wanted and I realised that the current way I was doing things was not going to allow me to make this change easily. I needed to refactor a fair amount to enable me to make the change less painful.
I didn't have any tests as this was a spike of development to poke around and see what was possible. The first thing I needed was some tests to ensure that I didn't break the current flow of things which I like.
It turns out that using rexml from the Ruby standard library is great because it allows me to not require any dependencies, which makes the workflow easier for people to install. The downside is that this library seems pretty clumsy to use and it seemed to spit out it's XML in a different order for each subsequent run so I couldn't test for equality on the string output. Both of these are bad qualities during testing.
I was not concerned about what gems I used in development as long as the final workflow doesn't make use of them so a short google later and I found a gem called lorax that allows you to perform diffs on xml using nokogiri. Armed with this gem I set about writing some functional tests to cover the existing use cases, which would allow me to freely refactor.
Too many responsibilities
In the original implementation the classes that represented the search results knew too much. They:
Knew a lot about where they should look for files on the file system - they provided their own glob pattern
Knew the file location of the simulator directory ~/Library/Application Support/iPhone Simulator
Instances knew how to represent themselves as XML
Instances knew how to extract metadata from their path
It was a combination of the first 2 items that meant that the change wasn't straight forward (would have required some nasty nasty code to get things working). The 3rd item I have a plan to take care of but it wasn't required for me to do the feature.
It turns out that both of the first 2 items can be taken care of with one fix - inject this information in.
The original implementation looked something like this
The flow here is
Line 3: Ask the concrete class SimulatorPath for the glob pattern (lines 11-13) to use for finding files and loop over the results
Line 4-5: Add any paths meeting the search criteria to the results
Line 7: Add a default item to the results if no result found
The new implementation looks more like this
This looks similar but the key thing is that it has all been abstracted to outside of the SimulatorResult class.
With that refactoring done and all tests green the change I wanted to actually implement was easy.
I created a new class called ApplicationResult that slightly tweaks the behavior from SimulatorApplicationResult it's superclass (great naming I know).
With the new class in place I just updated the ugly conditional used to decide which kind of query the user is performing and which result class that maps to.
This is wrapped in a handy struct to keep the information together. The sturct knows what glob to use for getting the file names on disk and then what class to send each returned path to with the correct query part.
The net result is that if the first character of your search is not 0..9 then this new class will be used. The result on screen is only slightly different but the usability is much improved.
The visible difference is that if you have drilled down into a simulator then the results show just the name of each of the returned apps. If you have not drilled down then each result is prefixed with the simulator version number so it's always easy to distinguish what you are looking at with minimal duplication in the UI.
Effects of iOS Dev Weekly
I put up my Alfred 2 Beta workflow called sidir and didn't really expect anyone to be interested as it was just a personal tool. It turns out that being mentioned on iOS Dev Weekly can get you a fair few extra eyes to look at your stuff. Take a look at the stats for visitors to this site in the image below, if you look close enough you might be able to see the influence that being mentioned on iOS Dev Weekly has. Thanks Dave.
This new workflow approach makes the process of working with the apps installed in your simulator directory a snap.
Bring up Alfred v2 and type sidir
The returned hits will only contain simulators that actually have apps installed.
You can use the keyboard shortcuts or just tab complete to the correct simulator to make the search results show a list of installed apps for that simulator version.
Simply hitting return on the app you want or using the keyboard shortcuts will open Finder at the sandbox for that application.
Yes that's right no more being confronted with this mess trying to find the app you are working on
As an added bonus, you can hold alt whilst accepting a search result in Alfred and it will delete the sandbox. This uses some Applescript so it will be moved to the Trash and not clobber any files with the same name, which is always nice.