Meta macros are pretty nifty but trying to follow how they work can really challenge the limits of your mental stack frame.
Today I’m going to try and follow macro expansion from start to finish using libextobjc to do some basic metaprogramming. This is going to be a bumpy ride but as always it’s great to see how different techniques can be used to solve problems.
Let’s start with a fictitious problem that I would like to solve with some metaprogramming. I would probably never do this in a real project but it gives me a realistic use case to work through.
Imagine I want my view controllers to fail hard if I forget to connect up an outlet in the xib file. I could start with something like this:
whoa that’s a lot of repetition and it’s not going to scale well. What would be great is if I could write some code that would write this repetitious code for me, ideally I would just type something like this:
After reading the docs I can see that the MACRO argument should be the name of a macro that takes two arguments in the form of MACRO(INDEX, ARG). The INDEX parameter will be the index of the current iteration in the for loop and the ARG parameter will be the argument for the current iteration in the for loop.
So I need to start of by defining a macro that takes these two arguments and expands to the NSParameterAssert that I want. Here’s a first stab at such a macro
I don’t actually care to use the value of INDEX so it is ignored. This is the macro that will be used within the metamacro_foreach and will eventually expand into the required NSParameterAsserts.
In each of the following examples I’ll show the input (starting macro) above the 3 dashes and what this would theoretically expand into below the 3 dashes. I’ll optionally show any macro definitions at the top of the code block.
##metamacro_foreach
Now let’s see how we can use metamacro_foreach to write the PASAssertConnections macro that will take in a list of ivar names and expand them to the required NSParameterAsserts.
In this case I pass OUTLET_ASSERT as the macro to use on each iteration. I pass ; to use as a separator between iterations, which will terminate each NSParameterAssert. Then finally a comma separated list of ivar names that we are going to iterate over and generate the NSParameterAsserts for.
metamacro_concat is the easier of the two so we’ll take a look at that first.
##metamacro_concat
#define metamacro_concat(A, B) \
metamacro_concat_(A, B)
#define metamacro_concat_(A, B) A ## B
metamacro_concat(metamacro_foreach_cxt,2)---metamacro_foreach_cxt2
Cool so metamacro_concat just expands to metamacro_concat_, which then just joins the tokens together using ##. So metamacro_concat just has the effect of joining it’s two arguments into one string.
The metamacro_argcount macro uses another macro called metamacro_at. The metamacro_at is similar in concept to indexing into an array like myArray[index]. In plain English this macro is the same as “give me the nth item in the following list”.
The metamacro_argcount macro uses a clever little trick. If we put the numbers from INDEX down to 0 into an array and then ask for the value at INDEX we would get the last number, which would be 0. If we preprend something to the beginning of this array and asked for the value at INDEX again we would now get 1.
Let’s see this in Objective-C so it’s easier to picture:
The relationship is that when you prepend an argument to the array you shift all of the numeric values to the right by one step, which moves a higher number into the index that is being fetched. This of course only works up to the value of INDEX - so we can tell that this particular implementation of metamacros only supports 20 arguments.
NB - this implementation of metamacros requires at least one argument to be given when using metamacro_argcount.
You’ll see the trick of inserting __VA_ARGS__ into argument lists at different points used a few times so it’s worth making sure you understand what is happening above.
Ok so that makes sense but what about metamacro_at?
The change is very subtle. The 20 has moved from being an argument to now actually being part of the macro name. So now we need to look up metamacro_at20
It turns out that there are variants of metamacro_at defined for 0 to 20, which allows you to access any of the first 20 arguments from the __VA_ARGS__ arguments.
This is another common trick you’ll see with metamacros, at some point you have to knuckle down and write out multiple versions of the same macro to handle different length argument lists. You’ll often see that metamacros are generated by other scripts that allow you to specify how many arguments you would like to support without having to hand roll all the variations of metamacro_at0..N.
To make metamacro_at a little easier to digest I’ll examine one of the smaller versions of this macro.
The _0 and _1 arguments are basically used as placeholders to gobble up the items at indices 0 and 1 from the arguments. Then we bundle the rest of the arguments together with .... The newly trimmed __VA_ARGS__ is then passed into metamacro_head
##metamacro_head
#define metamacro_head(...) \
metamacro_head_(__VA_ARGS__, 0)
#define metamacro_head_(FIRST, ...) FIRST
metamacro_head(passwordTextField,passwordConfirmationTextField)---passwordTextField
metamacro_head uses the opposite trick to metamacro_at*. In this case we are only interested in the first item and we want to throw away the rest of the __VA_ARGS__ list. This is achieved by grabbing the first argument in FIRST and then collecting the rest with ....
Wow that escalated quickly. We now need to unwind out mental stack frame back to metamacro_foreach_cxt.
Don’t worry the end is now very much in sight, just a couple more painless macro expansions. The previous expansion gives us the new metamacro_foreach_cxt2 macro to check out.
##metamacro_foreach_cxt2
This is another example of macro that has multiple versions defined from 0..20. Each of these foreach macros works by utilising the foreach macro that is defined to take one less argument than itself until we get all the way down to metamacro_foreach_cxt1
We are now at the point where we need to see what MACRO expands to. In this case MACRO is actually the metamacro_foreach_iter macro that we passed in near the beginning and I delayed explaining.
##metamacro_foreach_iter
This macro is really just an implementation detail and as such shouldn’t be used directly but we still want to see what part it plays:
Nice and simple - metamacro_foreach_iter is just a helper that takes our macro OUTLET_ASSERT and the two arguments that our macro should receive and puts the pieces in the right order to be further expanded into the NSParameterAssert calls that we want.
Thankfully that was only a minor detour so let’s get right back to metamacro_foreach_cxt2
And that’s it - we’ve followed the metamacro_foreach macro from the beginning of it’s use all the way to it’s end expansion and hopefully our heads are still in one piece.
##Wrapping up
At the beginning of the post I said I was aiming for
now I’m actually one step away from achieving this, but if this post has gotten your interest I’ll leave that as a simple exercise - it’s always better to learn by doing and not just skimming through blog posts hoping to learn by osmosis.
Metaprogramming is normally something that people associate with more dynamic languages like Ruby but there’s a whole load of possibilities and cool tricks out there just waiting to be learned. As always I encourage you to join me in peeling back the curtain and seeing that there is normally no magic to be found in your favorite OSS projects.
Follow me as I poke Expecta with a stick to see how it works - it’s pretty cool.
It’s been a long time since my last post but I thought I’d get stuck into how something that appears simple actually works. That subject will be Expecta matchers. If you don’t know what Expecta is then you might want to nip over to the Github repo and skim the Readme then we can get stuck in.
Take the following example:
expect(2).to.equal(2);
This line looks so simple but it’s hiding a lot of clever techniques that may or may not be useful to keep within your own development bag of tricks.
Let’s break this apart and demystify what’s really going on.
##expect
expect() looks like a function call, so you may assume that there is a function declared somewhere called expect. Thankfully that is not the case or this blog post would be very boring - instead we find this defined as:
_EXP_expect is the actual function we was looking for.
At this point you may be wondering why did they bother with this odd chain of macro expansions. The logical reason would most likely be that expect(id actual) is an optional short hand syntax, which is only enabled by defining EXP_SHORTHAND before importing Expecta.h. Without this define you have to use the long hand EXP_expect(id actual) and this is what expands to _EXP_expect with all the additional arguments.
Go ahead and reread that last paragraph a few times if it didn’t sink in the first time.
In effect the define for expect saves you from having to type out
The _EXP_expect function simply creates a new instance of EXPExpect with all the arguments supplied. I’m not going to go over the EXPExpect class as I want to cover the single line of code at the top of this post.
Before we move on though it’s worth pointing out that Expecta is really cool as it does not require you to box your arguments. It’s the EXPObjectify function that does the work of making sure that if you pass in a primitive like int, float, double, etc then it will box it with an NSValue or NSNumber automatically for you.
##to
to looks simple enough - so why is this interesting? Well knowing that it is used like this to.equal... leads us to the conclusion that it returns an instance of something that responds to equal. Before following the link to look up the definition keep in mind that to is entirely optional and I could validly call expect(2).equal(2); - this should narrow down what to returns.
@property(nonatomic,readonly)EXPExpect*to;
Yup you may have guessed it to returns an instance of EXPExpect - not just any instance but the instance it was called on - check it out:
-(EXPExpect*)to{returnself;}
This is just a little sprinkling of syntactic sugar. It makes the expectation read better consider expect(2).to.equal(2) vs expect(2).equal(2).
##equal
By now you are probably thinking that the interesting stuff is over and equal will just be a property declared on EXPExpect. You may also come to the conclusion that the property will return a pointer to either a block or a function so that it can be invoked with parentheses and an argument equal(2). This is exactly what is happening - kind of…
If you search the EXPExpect class you will not find a property declaration but if you follow the declarationf of equal through you’ll land in EXPExpect+equal.h, which looks like this:
#import "Expecta.h"
EXPMatcherInterface(_equal,(idexpected));EXPMatcherInterface(equal,(idexpected));// to aid code completion#define equal(...) _equal(EXPObjectify((__VA_ARGS__)))
This is where we have to make sure our brain is really engaged and step up a gear. Take a breather and join me after the relaxing grey line…
In english this has declared a named category called _equalMatcher on the EXPExpect class. This category declares a single readonly property, which means that in the .m file we would expect to see a single method declared with the signature - (void(^)(id expected))_equal;
NB I showed the mapping of _equal as equal is only used for code completion and there is in fact never an implementation declared for - (void(^)(id expected))equal;
So being inquisitive we jump to the implementation to see how this method is defined and we find more #define magic.
EXPMatcherImplementationBegin(_equal,(idexpected)){match(^BOOL{if((actual==expected)||[actualisEqual:expected]){returnYES;}elseif([actualisKindOfClass:[NSNumberclass]]&&[expectedisKindOfClass:[NSNumberclass]]){if(EXPIsNumberFloat((NSNumber*)actual)||EXPIsNumberFloat((NSNumber*)expected)){return[(NSNumber*)actualfloatValue]==[(NSNumber*)expectedfloatValue];}}returnNO;});failureMessageForTo(^NSString*{return[NSStringstringWithFormat:@"expected: %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];});failureMessageForNotTo(^NSString*{return[NSStringstringWithFormat:@"expected: not %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];});}EXPMatcherImplementationEnd
When the two macros are expanded we end up with this (formatting mine):
If we didn’t want to use the macros (not advised at all - only shown for interest sake) then the implementation could remove the added complexity of setting up the DSL and end up with an implementation like this:
-(void(^)(idexpected))_equal{__blockidactual=self.actual;return[[^(idexpected){EXPBlockDefinedMatcher*matcher=[[EXPBlockDefinedMatcheralloc]init];matcher.matchBlock=^BOOL{if((actual==expected)||[actualisEqual:expected]){returnYES;}elseif([actualisKindOfClass:[NSNumberclass]]&&[expectedisKindOfClass:[NSNumberclass]]){if(EXPIsNumberFloat((NSNumber*)actual)||EXPIsNumberFloat((NSNumber*)expected)){return[(NSNumber*)actualfloatValue]==[(NSNumber*)expectedfloatValue];}}returnNO;};matcher.failureMessageForToBlock=^NSString*{return[NSStringstringWithFormat:@"expected: %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];};matcher.failureMessageForNotToBlock=^NSString*{return[NSStringstringWithFormat:@"expected: not %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];};[selfapplyMatcher:matcherto:&actual];}copy]autorelease];}
This version seems like an awful lot of error prone boiler plate code that a developer would have to write for each matcher. Keep in mind that there are ~25 matchers included with Expecta and you can define your own.
Let’s list the required steps for this implementation:
Configuring this instance’s properties with blocks for matchBlock (line 7), failureMessageForToBlock (line 18), and failureMessageForNotToBlock (line 22).
Ensures that [self applyMatcher:matcher to:&actual]; is executed at the end of the block that is declared and returned (line 3).
In this implementation there are only 3 steps so this must surely be better?
Nope:
In the original list of 5 steps only step 4 was exposed to the developer and the remaining steps were hidden behind macros.
In this implementation the developer has to know about all 3 steps.
This means that the developer has 2 extra steps to remember to do and to make matters worse they are wrapping (before + after) steps.
In the first listing the developer is literally just stating the test requirements, whereas in this version the developer has to know about matchers and how they need to be configured in addition to the test requirements.
So hopefully now I’ve pulled back the curtain a little the line
expect(2).to.equal(2);
won’t seem as mysterious.
##Wrapping up
Well that was a heavy post with a lot to understand and I do apologise if I got any of it wrong - I’m no Expecta expert. Reading code is great fun especially when you get that Eureka moment and you learn something new. The joy of a project like Expecta is that it is all unit tested so you can hack around and change things to test your assumptions - hit test and wait for your theory to be validated with a sea of red or green unit test results.
Storyboards can really speed up development but a problem I run into over and over again is working with identifiers that are set in the storyboard and referenced in code.
#Problem
Let’s take the example of making a segue:
You create a segue and then assign it an identifier in the storyboard… and that’s the problem. The identifier is defined in the storyboard, which leaves me with plenty of opportunities to mess up referencing the segue in code.
My normal strategy is to create a constants file that would have a constant with the value of each of these identifiers, which starts to look like the following (forgive the naming):
This works for me and I’m happy, but it seems like a lot of manual graft and can become tiresome to maintain especially in a young project with lots of development spikes. It’s also still very possible to make typos and mess up these declarations or for them to become out of sync with the storyboard.
#Automation
I got to thinking about this problem and two things stood out
I don’t like doing things manually and
I don’t like that it is so easy to misspell a constant when swapping between storyboard and code (yes I am aware copy and paste exists).
So I started tinkering and I’ve come up with a rough tool that I’ve cunningly named sbconstants.
My automation idea is that you add a build script that will extract constants from your storyboards and then dump them into a file.
Grab it with
sudo gem install sbconstants
My first effort has a UI like this (command line app)
% sbconstants -h
Usage: DESTINATION_FILE [options]
-p, --prefix=<prefix> Only match identifiers with <prefix>
-s, --source-dir=<source> Directory containing storyboards
-q, --queries=<queries> YAML file containing queries
-d, --dry-run Output to STDOUT
-v, --verbose Verbose output
To get this working on a project the sbconstantsgem needs to be installed to the system Ruby and then we need to call the executable from a build script. An example call would look like this
NB The argument is the destination file to dump the constants into - this needs to be added manually
Every time this project is built it will parse the storyboard files and pull out any constants and then dump them into PASStoryboardConstants.(h|m). This of course means that PASStoryboardConstants.(h|m) should not be edited by hand as it will just be clobbered any time we build.
The output of running this tool will look something like this:
PASStoryboardConstants.h
// Auto generated file - any changes will be lost#pragma mark - tableViewCell.reuseIdentifier
externNSString*constPSBAwesomeCell;#pragma mark - segue.identifier
externNSString*constPSBMasterToDetail;externNSString*constPSBMasterToSettings;
PASStoryboardConstants.m
// Auto generated file - any changes will be lost#import "PASStoryboardConstants.h"
#pragma mark - tableViewCell.reuseIdentifier
NSString*constPSBAwesomeCell=@"PSBAwesomeCell";#pragma mark - segue.identifier
NSString*constPSBMasterToDetail=@"PSBMasterToDetail";NSString*constPSBMasterToSettings=@"PSBMasterToSettings";
The constants are grouped by where they were found in the storyboard xml e.g. segue.identifier. This can really help give you some context about where/what/when and why a constant exists.
##More options
Options are fun and there are a few to play with - most of these options are really only any good for debugging.
###--prefix
-p, –prefix= Only match identifiers with
Using the prefix option you can specify that you only want to grab identifiers that start with a certain prefix, which is always nice.
If you don’t want to run the tool from the root of your app for some reason you can specify the source directory to start searching for storyboard files. The search is recursive using a glob something like <source-dir>/**/*.storyboard
###--dry-run
-d, –dry-run Output to STDOUT
If you just want to run the tool and not write the output to a file then this option will spit the result out to $stdout
###--verbose
-v, –verbose Verbose output
Perhaps you want a little more context about where your identifiers are being grabbed from for debugging purposes. Never fear just use the --verbose switch and get output similar to:
sample output
#pragma mark - viewController.storyboardIdentifier////info: MainStoryboard[line:43](viewController.storyboardIdentifier)//context: <viewControllerrestorationIdentifier="asd"storyboardIdentifier="FirstViewController"id="EPD-sv-vrF"sceneMemberID="viewController">//NSString*constFirstViewController=@"FirstViewController";
Chances are I’ve missed some identifiers to search for in the storyboard. You don’t want to wait for the gem to be updated or have to fork it and fix it. Using this option you can provide a YAML file that contains a description of what identifers to search for. The current one looks something like this (NB this is a great starting point for creating your own yaml):
This looks a little funky but it’s essentially groups of keys and values (both the key and the value can be an array). This actually gets expanded to looks for
I’m not sure if this tool will be of any use to anyone but for the projects I’ve been tinkering with it seems to work pretty well leaving me confident that my storyboard identifier’s will not go out of sync with how they are referenced in code.
If anyone does use this and they like it or they have any issues please report it to me so I can get it fixed up and improve my own workflow.