I explore how bitwise manipulations allow you to store the red, green, blue and alpha components of a colour in a single integer.
Today I’m going to step through a helper method that allows you to create a UIColor instance from the hex colour values that designers love to give you. The category method in question looks like this:
+(UIColor*)pas_colorWithRGBAHex:(u_int32_t)rgba;
When I first saw an implementation of this many years ago I remember staring at the source and feeling my eyes glaze over. Let’s break this apart and demystify what’s really going on.
##u_int32_t
One of the questions you might ask when looking at this method is why be so specific with the type of the parameter and how do we know it needs to be a u_int32_t and not something else like u_int64_t or int? To answer that we’ll look at some sample input provided by a designer in the form of a lovely green colour.
#01A902FF /* a lovely green */
This is actually 4 pairs of hexadecimal numbers, reading from left to right we have:
The Red component is #01
The Green component is #A9
The Blue component is #02
The Alpha component is #FF
In base 10, our normal counting system, this would be 1, 169, 2, 255 respectively.
Each hexadecimal component covers the range 00..FF, which is equivalent to 0..255. To be able to represent all the numbers from 0..255 in binary we would need 8 bits.
####bits
A bit can only represent 21 values as it can either be 1 or 0. By looking at groups of bits we can increase the number of values that can be represented. For example a 4 bit number can be configured in 24 or 2 x 2 x 2 x 2 different combinations, which means we can use it to represent 16 unique values.
To represent 0..255 we would need 8 bits as 28 or 2 x 2 x 2 x 2 x 2 x 2 x 2 x 2 equals 256.
So knowing that we have to represent 4 colour components that each require 8 bits we can finally understand why we chose u_int32_t as the type of the argument (4 x 8 = 32).
##rgba
The next logical question is how do we extract the four 8 bit values from a single integer?
Let’s start by looking at how our hexadecimal #01A902FF (green) value actually looks under the hood when represented by a u_int32_t. Keep in mind that a u_int32_t is just a collection of 32 bits that can each have a value of 1 or 0:
1101010010000001011111111 // binary solo (Flight of the Conchords reference)
Let’s do that again but with some spacing and some markers to show which bits represent our RGBA components.
+-- R --+-- G --+-- B --+-- A --+
| | | | |
0000 0001 1010 1001 0000 0010 1111 1111
When a bit is set to 1 it means that the unit that the bit corresponds to should be included in the total. The units are powers of 2, with the power to raise by incrementing by 1 each time and starting from the right with 20. Here’s a few examples using just the first 8 bits:
With that slight detour out of the way we can crack on with extracting the Alpha component, which is the easiest to work with. Currently our u_int32_t is capable of representing the values 0..4294967295, but we only want to extract an 8 bit value with a range of 0..255. We need to somehow look at just the 8 bits that we care about - this can be achieved with bit masking.
##&
The logical AND, which is represented by the single ampersand character (&), is used to compare two values by looking at the bits that the values are represented by. When using the logical AND a bit is only set to 1 in the resulting value if it is 1 in both the left and right values used in an expression.
15 is represented by the binary 0b1111. So 15 & 15 would equal 15 and look like this:
0b1111 /*
0b1111 & * All the bits are on in both values so all
------ * of the bits in the resulting value are on
0b1111 */
0 is represented by the binary 0b0000. So 0 & 15 would equal 0 and look like this:
0b0000 /*
0b1111 & * There are no bits that are on in both values
------ * so the resulting value has no bits on
0b0000 */
10 is represented by the binary 0b1010 and 12 looks like 0b1100. So 10 & 12 would result in 8 and look like this:
0b1010 /*
0b1100 & * Only the 8 bit is on in both values so only
------ * the 8 bit is on in the result
0b1000 */
This may not seem very helpful but this is the key to being able to treat our 32 bit value as if it was only 8 bits. To get our Alpha component we need to turn of the 24 bits that we don’t care about and ensure that the first 8 bits keep their current values.
To achieve this we use a bit pattern that turns on only the first 8 bits and then logical AND this with our RGBA value:
rgba & #000000FF
#000000FF is hex for 255, which is the first 8 bits turned on (0b1111 1111) - here is how the above looks at the bit level:
+-- R --+-- G --+-- B --+-- A --+
| | | | |
0000 0001 1010 1001 0000 0010 1111 1111
0000 0000 0000 0000 0000 0000 1111 1111 &
---------------------------------------
0000 0000 0000 0000 0000 0000 1111 1111
Great this has masked out the last 24 bits of our u_int32_t and has given us the result of 255 for the Alpha component.
Hopefully this makes sense and we are ready to try and extract more components. Let’s jump to the Green value for a challenge. With our new technique we may naively assume that we just want to mask out all of the data that we don’t want. We could do that with the following:
rgba & #00FF0000
#00FF0000 is the same as 0b0000 0000 1111 1111 0000 0000 0000 0000 - at the bit level we get:
+-- R --+-- G --+-- B --+-- A --+
| | | | |
0000 0001 1010 1001 0000 0010 1111 1111
0000 0000 1111 1111 0000 0000 0000 0000 &
---------------------------------------
0000 0000 1010 1001 0000 0000 0000 0000
This has indeed masked out the data we don’t care about, but the resulting value is 11075584 which is well beyond the 0..255 range we was looking for. This is because the bits that are set to 1 represent 65536 + 524288 + 2097152 + 8388608.
What we need to do is shift the data to the right so that the bits representing the Green component move from their current place, in the middle, all the way over to the right so that they are in the first 8 bits of our variable. If the Green bit pattern (1010 1001) occurred in the first 8 bits we would get 1 + 8 + 32 + 128 = 169.
We move the values to the right with the right shift operator.
##>>
We want to shift the 8 bits that represent the Green component all the way to the right. There are 16 bits in front of the Green component so we need to shift our number 16 times to the right - simple
rgba >> 16
As the bits are shifted to the right they will fall of the right hand side, then new values will need to be added to the left to pad the newly created space. What the padding values are depends on the variable’s type and a few other things. After doing rgba >> 16 our bits now look like this
+------ new ------+-- R --+-- G --+
| | | |
0000 0000 0000 0000 0000 0000 1010 1001
This has gotten the bits of the Green component into a position where we can just mask them off like we did with the Alpha component.
If we follow this logic for each component we end up with the final demystified implementation of:
+ (UIColor *)pas_colorWithRGBAHex:(u_int32_t)rgba;
{
int r = 0xFF & (rgba >> 24);
int g = 0xFF & (rgba >> 16);
int b = 0xFF & (rgba >> 8);
int a = 0xFF & (rgba);
return [UIColor colorWithRed:r / 255.0f
green:g / 255.0f
blue:b / 255.0f
alpha:a / 255.0f];
}
For each component we follow the pattern of:
Shift the bits in the rgba variable to the right until the data we are interested in is in the first 8 bits
Mask of the first 8 bits to ensure that any other bits are ignored
This category method requires the hex value to be defined in the form of 2 hexadecimal numbers for each colour component - you could easily make different methods to handle different formats now you know how it all works. This category method would be used like this:
This post only looks at the right shift >> and logical AND & operators but hopefully it allows you to begin to imagine in your minds eye how bits are stored and can be manipulated.
Essentially under the hood the u_int32_t variable is just held as a sequence of bits. We are exploiting this structure to encode more than one piece of information in a single variable. We then have to use various bitwise operations to extract the meaning that we encoded into the integer.
Meta macros are pretty nifty but trying to follow how they work can really challenge the limits of your mental stack frame.
Today I’m going to try and follow macro expansion from start to finish using libextobjc to do some basic metaprogramming. This is going to be a bumpy ride but as always it’s great to see how different techniques can be used to solve problems.
Let’s start with a fictitious problem that I would like to solve with some metaprogramming. I would probably never do this in a real project but it gives me a realistic use case to work through.
Imagine I want my view controllers to fail hard if I forget to connect up an outlet in the xib file. I could start with something like this:
whoa that’s a lot of repetition and it’s not going to scale well. What would be great is if I could write some code that would write this repetitious code for me, ideally I would just type something like this:
After reading the docs I can see that the MACRO argument should be the name of a macro that takes two arguments in the form of MACRO(INDEX, ARG). The INDEX parameter will be the index of the current iteration in the for loop and the ARG parameter will be the argument for the current iteration in the for loop.
So I need to start of by defining a macro that takes these two arguments and expands to the NSParameterAssert that I want. Here’s a first stab at such a macro
I don’t actually care to use the value of INDEX so it is ignored. This is the macro that will be used within the metamacro_foreach and will eventually expand into the required NSParameterAsserts.
In each of the following examples I’ll show the input (starting macro) above the 3 dashes and what this would theoretically expand into below the 3 dashes. I’ll optionally show any macro definitions at the top of the code block.
##metamacro_foreach
Now let’s see how we can use metamacro_foreach to write the PASAssertConnections macro that will take in a list of ivar names and expand them to the required NSParameterAsserts.
In this case I pass OUTLET_ASSERT as the macro to use on each iteration. I pass ; to use as a separator between iterations, which will terminate each NSParameterAssert. Then finally a comma separated list of ivar names that we are going to iterate over and generate the NSParameterAsserts for.
metamacro_concat is the easier of the two so we’ll take a look at that first.
##metamacro_concat
#define metamacro_concat(A, B) \
metamacro_concat_(A, B)
#define metamacro_concat_(A, B) A ## B
metamacro_concat(metamacro_foreach_cxt,2)---metamacro_foreach_cxt2
Cool so metamacro_concat just expands to metamacro_concat_, which then just joins the tokens together using ##. So metamacro_concat just has the effect of joining it’s two arguments into one string.
The metamacro_argcount macro uses another macro called metamacro_at. The metamacro_at is similar in concept to indexing into an array like myArray[index]. In plain English this macro is the same as “give me the nth item in the following list”.
The metamacro_argcount macro uses a clever little trick. If we put the numbers from INDEX down to 0 into an array and then ask for the value at INDEX we would get the last number, which would be 0. If we preprend something to the beginning of this array and asked for the value at INDEX again we would now get 1.
Let’s see this in Objective-C so it’s easier to picture:
The relationship is that when you prepend an argument to the array you shift all of the numeric values to the right by one step, which moves a higher number into the index that is being fetched. This of course only works up to the value of INDEX - so we can tell that this particular implementation of metamacros only supports 20 arguments.
NB - this implementation of metamacros requires at least one argument to be given when using metamacro_argcount.
You’ll see the trick of inserting __VA_ARGS__ into argument lists at different points used a few times so it’s worth making sure you understand what is happening above.
Ok so that makes sense but what about metamacro_at?
The change is very subtle. The 20 has moved from being an argument to now actually being part of the macro name. So now we need to look up metamacro_at20
It turns out that there are variants of metamacro_at defined for 0 to 20, which allows you to access any of the first 20 arguments from the __VA_ARGS__ arguments.
This is another common trick you’ll see with metamacros, at some point you have to knuckle down and write out multiple versions of the same macro to handle different length argument lists. You’ll often see that metamacros are generated by other scripts that allow you to specify how many arguments you would like to support without having to hand roll all the variations of metamacro_at0..N.
To make metamacro_at a little easier to digest I’ll examine one of the smaller versions of this macro.
The _0 and _1 arguments are basically used as placeholders to gobble up the items at indices 0 and 1 from the arguments. Then we bundle the rest of the arguments together with .... The newly trimmed __VA_ARGS__ is then passed into metamacro_head
##metamacro_head
#define metamacro_head(...) \
metamacro_head_(__VA_ARGS__, 0)
#define metamacro_head_(FIRST, ...) FIRST
metamacro_head(passwordTextField,passwordConfirmationTextField)---passwordTextField
metamacro_head uses the opposite trick to metamacro_at*. In this case we are only interested in the first item and we want to throw away the rest of the __VA_ARGS__ list. This is achieved by grabbing the first argument in FIRST and then collecting the rest with ....
Wow that escalated quickly. We now need to unwind out mental stack frame back to metamacro_foreach_cxt.
Don’t worry the end is now very much in sight, just a couple more painless macro expansions. The previous expansion gives us the new metamacro_foreach_cxt2 macro to check out.
##metamacro_foreach_cxt2
This is another example of macro that has multiple versions defined from 0..20. Each of these foreach macros works by utilising the foreach macro that is defined to take one less argument than itself until we get all the way down to metamacro_foreach_cxt1
We are now at the point where we need to see what MACRO expands to. In this case MACRO is actually the metamacro_foreach_iter macro that we passed in near the beginning and I delayed explaining.
##metamacro_foreach_iter
This macro is really just an implementation detail and as such shouldn’t be used directly but we still want to see what part it plays:
Nice and simple - metamacro_foreach_iter is just a helper that takes our macro OUTLET_ASSERT and the two arguments that our macro should receive and puts the pieces in the right order to be further expanded into the NSParameterAssert calls that we want.
Thankfully that was only a minor detour so let’s get right back to metamacro_foreach_cxt2
And that’s it - we’ve followed the metamacro_foreach macro from the beginning of it’s use all the way to it’s end expansion and hopefully our heads are still in one piece.
##Wrapping up
At the beginning of the post I said I was aiming for
now I’m actually one step away from achieving this, but if this post has gotten your interest I’ll leave that as a simple exercise - it’s always better to learn by doing and not just skimming through blog posts hoping to learn by osmosis.
Metaprogramming is normally something that people associate with more dynamic languages like Ruby but there’s a whole load of possibilities and cool tricks out there just waiting to be learned. As always I encourage you to join me in peeling back the curtain and seeing that there is normally no magic to be found in your favorite OSS projects.
Follow me as I poke Expecta with a stick to see how it works - it’s pretty cool.
It’s been a long time since my last post but I thought I’d get stuck into how something that appears simple actually works. That subject will be Expecta matchers. If you don’t know what Expecta is then you might want to nip over to the Github repo and skim the Readme then we can get stuck in.
Take the following example:
expect(2).to.equal(2);
This line looks so simple but it’s hiding a lot of clever techniques that may or may not be useful to keep within your own development bag of tricks.
Let’s break this apart and demystify what’s really going on.
##expect
expect() looks like a function call, so you may assume that there is a function declared somewhere called expect. Thankfully that is not the case or this blog post would be very boring - instead we find this defined as:
_EXP_expect is the actual function we was looking for.
At this point you may be wondering why did they bother with this odd chain of macro expansions. The logical reason would most likely be that expect(id actual) is an optional short hand syntax, which is only enabled by defining EXP_SHORTHAND before importing Expecta.h. Without this define you have to use the long hand EXP_expect(id actual) and this is what expands to _EXP_expect with all the additional arguments.
Go ahead and reread that last paragraph a few times if it didn’t sink in the first time.
In effect the define for expect saves you from having to type out
The _EXP_expect function simply creates a new instance of EXPExpect with all the arguments supplied. I’m not going to go over the EXPExpect class as I want to cover the single line of code at the top of this post.
Before we move on though it’s worth pointing out that Expecta is really cool as it does not require you to box your arguments. It’s the EXPObjectify function that does the work of making sure that if you pass in a primitive like int, float, double, etc then it will box it with an NSValue or NSNumber automatically for you.
##to
to looks simple enough - so why is this interesting? Well knowing that it is used like this to.equal... leads us to the conclusion that it returns an instance of something that responds to equal. Before following the link to look up the definition keep in mind that to is entirely optional and I could validly call expect(2).equal(2); - this should narrow down what to returns.
@property(nonatomic,readonly)EXPExpect*to;
Yup you may have guessed it to returns an instance of EXPExpect - not just any instance but the instance it was called on - check it out:
-(EXPExpect*)to{returnself;}
This is just a little sprinkling of syntactic sugar. It makes the expectation read better consider expect(2).to.equal(2) vs expect(2).equal(2).
##equal
By now you are probably thinking that the interesting stuff is over and equal will just be a property declared on EXPExpect. You may also come to the conclusion that the property will return a pointer to either a block or a function so that it can be invoked with parentheses and an argument equal(2). This is exactly what is happening - kind of…
If you search the EXPExpect class you will not find a property declaration but if you follow the declarationf of equal through you’ll land in EXPExpect+equal.h, which looks like this:
#import "Expecta.h"
EXPMatcherInterface(_equal,(idexpected));EXPMatcherInterface(equal,(idexpected));// to aid code completion#define equal(...) _equal(EXPObjectify((__VA_ARGS__)))
This is where we have to make sure our brain is really engaged and step up a gear. Take a breather and join me after the relaxing grey line…
In english this has declared a named category called _equalMatcher on the EXPExpect class. This category declares a single readonly property, which means that in the .m file we would expect to see a single method declared with the signature - (void(^)(id expected))_equal;
NB I showed the mapping of _equal as equal is only used for code completion and there is in fact never an implementation declared for - (void(^)(id expected))equal;
So being inquisitive we jump to the implementation to see how this method is defined and we find more #define magic.
EXPMatcherImplementationBegin(_equal,(idexpected)){match(^BOOL{if((actual==expected)||[actualisEqual:expected]){returnYES;}elseif([actualisKindOfClass:[NSNumberclass]]&&[expectedisKindOfClass:[NSNumberclass]]){if(EXPIsNumberFloat((NSNumber*)actual)||EXPIsNumberFloat((NSNumber*)expected)){return[(NSNumber*)actualfloatValue]==[(NSNumber*)expectedfloatValue];}}returnNO;});failureMessageForTo(^NSString*{return[NSStringstringWithFormat:@"expected: %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];});failureMessageForNotTo(^NSString*{return[NSStringstringWithFormat:@"expected: not %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];});}EXPMatcherImplementationEnd
When the two macros are expanded we end up with this (formatting mine):
If we didn’t want to use the macros (not advised at all - only shown for interest sake) then the implementation could remove the added complexity of setting up the DSL and end up with an implementation like this:
-(void(^)(idexpected))_equal{__blockidactual=self.actual;return[[^(idexpected){EXPBlockDefinedMatcher*matcher=[[EXPBlockDefinedMatcheralloc]init];matcher.matchBlock=^BOOL{if((actual==expected)||[actualisEqual:expected]){returnYES;}elseif([actualisKindOfClass:[NSNumberclass]]&&[expectedisKindOfClass:[NSNumberclass]]){if(EXPIsNumberFloat((NSNumber*)actual)||EXPIsNumberFloat((NSNumber*)expected)){return[(NSNumber*)actualfloatValue]==[(NSNumber*)expectedfloatValue];}}returnNO;};matcher.failureMessageForToBlock=^NSString*{return[NSStringstringWithFormat:@"expected: %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];};matcher.failureMessageForNotToBlock=^NSString*{return[NSStringstringWithFormat:@"expected: not %@, got: %@",EXPDescribeObject(expected),EXPDescribeObject(actual)];};[selfapplyMatcher:matcherto:&actual];}copy]autorelease];}
This version seems like an awful lot of error prone boiler plate code that a developer would have to write for each matcher. Keep in mind that there are ~25 matchers included with Expecta and you can define your own.
Let’s list the required steps for this implementation:
Configuring this instance’s properties with blocks for matchBlock (line 7), failureMessageForToBlock (line 18), and failureMessageForNotToBlock (line 22).
Ensures that [self applyMatcher:matcher to:&actual]; is executed at the end of the block that is declared and returned (line 3).
In this implementation there are only 3 steps so this must surely be better?
Nope:
In the original list of 5 steps only step 4 was exposed to the developer and the remaining steps were hidden behind macros.
In this implementation the developer has to know about all 3 steps.
This means that the developer has 2 extra steps to remember to do and to make matters worse they are wrapping (before + after) steps.
In the first listing the developer is literally just stating the test requirements, whereas in this version the developer has to know about matchers and how they need to be configured in addition to the test requirements.
So hopefully now I’ve pulled back the curtain a little the line
expect(2).to.equal(2);
won’t seem as mysterious.
##Wrapping up
Well that was a heavy post with a lot to understand and I do apologise if I got any of it wrong - I’m no Expecta expert. Reading code is great fun especially when you get that Eureka moment and you learn something new. The joy of a project like Expecta is that it is all unit tested so you can hack around and change things to test your assumptions - hit test and wait for your theory to be validated with a sea of red or green unit test results.