Git apply-edit - improving `git edit`

16 Jun 2019

In my last post I discussed the creation of a git helper called git-edit. The subcommand is really helpful but in this short post I’m going to look at a slightly different way of adding new git functionality that is built on top of git-edit.

Problem outline

When working on a feature I often find myself writing code and thinking “this should really be in an earlier commit”. There are many reasons for doing this:

It might logically makes more sense for some work to appear in a certain order
I might opt to move some work earlier and cut a shorter branch so other features can use code earlier
- e.g. I might aim to get a few commits merged quickly whilst the rest of the feature is being fleshed out
I might want to add a “REMOVE ME” commit
- If I need to add a temporary hack to allow some feature work I put it in an isolated commit as soon as possible. Being in a different commit makes it really simple to delete the commit before pushing for code review

These are all good uses for git-edit. A repeating usage pattern I see in my own work is that I will create the new work and then realise that I don’t want to target head, instead I want to target an earlier commit. So to perform a git-edit I often need to stash my changes, then run git-edit followed by git stash pop. Looking in my zsh history it appears that I do this a lot so it would be useful to automate it a bit.

git alias

For a simple chain of commands like this I don’t really need to make a separate executable script as I can just leverage git’s ability to add aliases.

~/.gitconfig

[alias]
  apply-edit = !sh -c 'git stash && git edit $1 && git stash pop' -

With this alias added to my .gitconfig I can now run git apply-edit <some-sha> from my current position.

I like this flow and the fact that the pop will fail if the working directory can not be reproduced cleanly just reminds me that if I had just used a --fixup this would have been more painful and less immediate to resolve.

Git edit - improving --fixup

14 Jun 2019

I’m a big fan of attempting to make my git history useful. This often means reworking my commits so that work is done in a sensible order and in small logical units. There are many ways to rework your history in git but one I’ve used a lot for the past year is the --fixup flag when committing. This has often led to annoying problems, in this post I’ll look at:

Using --fixup
Problems with --fixup
A solution git-edit
- Creating your own git commands
- GIT_SEQUENCE_EDITOR
Conclusion

Using `--fixup`

The --fixup flag is a bit like the --amend flag except you can specify the commit you want to squash your work into rather than it just being squashed into the most recent commit.

When given the following history:

* 0230dcb (HEAD -> master) Add git-edit post
* 4564152 Add testing tips post
* 9051216 Add tying things together post

If I stage some changes and run git commit --amend then the last commit Add git-edit post will be rewritten to include my staged changes.

If instead I want my changes to go into a commit that is not the most recent then I can use the following command git commit --fixup 4564152. In this example I would end up with the following

* a5f2563 (HEAD -> master) fixup! Add testing tips post
* 0230dcb Add git-edit post
* 4564152 Add testing tips post
* 9051216 Add tying things together post

Once I have this I can run git rebase --interactive 9051216 and accept the todo list to have git squash the new changes into the target commit.

Problems with `--fixup`

I love --fixup but it doesn’t quite do so well when you have heavy churn in your files (possibly a sign that the work could be reordered). The issue you run into a lot is that when you make your changes to fix up a commit you are potentially looking at a source file that has been changed in multiple steps. Any changes you make may only make sense in the current state of the file, this can lead to conflicts when targeting an earlier revision of the file.

A solution `git-edit`

We can avoid the issues above by:

rolling back our repo to the target commit
applying the changes we want
reapply the remaining commits

This is achievable by using our friend git rebase --interactive <commitish>. Going back to our earlier example

* 0230dcb (HEAD -> master) Add git-edit post
* 4564152 Add testing tips post
* 9051216 Add tying things together post

If I want to make changes to the commit 4564152 then I can run git rebase --interactive 4564152~1 to be presented with a todo-list in an editor.

pick·4564152·Add·testing·tips·post¬
pick·0230dcb·Add·git-edit·post¬
¬
#·Rebase·9051216..0230dcb·onto·9051216·(2·commands)¬
#¬
#·Commands:¬
#·p,·pick·<commit>·=·use·commit¬
#·r,·reword·<commit>·=·use·commit,·but·edit·the·commit·message¬
#·e,·edit·<commit>·=·use·commit,·but·stop·for·amending¬
#·s,·squash·<commit>·=·use·commit,·but·meld·into·previous·commit¬
#·f,·fixup·<commit>·=·like·"squash",·but·discard·this·commit's·log·message¬
#·x,·exec·<command>·=·run·command·(the·rest·of·the·line)·using·shell¬
#·b,·break·=·stop·here·(continue·rebase·later·with·'git·rebase·--continue')¬
#·d,·drop·<commit>·=·remove·commit¬
#·l,·label·<label>·=·label·current·HEAD·with·a·name¬
#·t,·reset·<label>·=·reset·HEAD·to·a·label¬
#·m,·merge·[-C·<commit>·|·-c·<commit>]·<label>·[#·<oneline>]¬
#·.·······create·a·merge·commit·using·the·original·merge·commit's¬
#·.·······message·(or·the·oneline,·if·no·original·merge·commit·was¬
#·.·······specified).·Use·-c·<commit>·to·reword·the·commit·message.¬
#¬
#·These·lines·can·be·re-ordered;·they·are·executed·from·top·to·bottom.¬
#¬
#·If·you·remove·a·line·here·THAT·COMMIT·WILL·BE·LOST.¬
#¬
#·However,·if·you·remove·everything,·the·rebase·will·be·aborted.¬
#¬
#·Note·that·empty·commits·are·commented·out¬

I can now change the first pick line (which is for the commit we passed in) to say edit instead e.g.

- pick·4564152·Add·testing·tips·post¬
+ edit 4564152·Add·testing·tips·post

After saving this file I’ll be placed on the commit 4564152 where I can make my edits, add changes and finish up by calling git rebase --continue.

The above is fine but it’s a little manual so it would be nice if we could automate away some of these steps. Let’s see how we can do that.

Creating your own git commands

The first piece of the puzzle is creating a git command. I’d like the command to work as if it was a git subcommand by typing git edit <commitish>. This is actually not too difficult to do - add an executable command somewhere on your path with the name git-edit. Now when I call git edit <commitish> git will invoke my script. Let’s test it

~/bin/git-edit

echo "hey"

zsh

> git edit aabbcc
hey

`GIT_SEQUENCE_EDITOR`

The next challenge is how do I automate editing the todo list that you get when you call git edit? Again git has a nice seam for us to work with - we can set the environment variable for GIT_SEQUENCE_EDITOR to our own command. Our command will be invoked with one argument, the path to the todo-list file.

With this knowledge we can write a script that will:

read a todo-list file
update the correct todo-list item to be edit instead of pick
save the file

Let’s do this in ruby:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/usr/bin/env ruby

if ENV['GIT_SEQUENCE_EDITOR'] != __FILE__
  exec "GIT_SEQUENCE_EDITOR='#{__FILE__}' git rebase --interactive #{ARGV.first}~1"
end

todo_list = ARGV.first

lines = File.readlines(todo_list)
  .lazy
  .take_while { |line| line != "\n" }
  .map        { |line| line.gsub("fixup", "pick") }
  .to_a

lines[0] = lines[0].gsub("pick", "edit")

File.write(todo_list, lines.join)

Lines 3-5 allow me to use a single script to handle editing the todo-list and creating my personal git edit subcommand
- When I call git edit <commitish> the GIT_SEQUENCE_EDITOR will not be equal to this executable script’s name so it will execute the interactive rebase command on line 4 and set the GIT_SEQUENCE_EDITOR to the current file
Lines 9-13 handle reading in the todo-list, stripping all comments from the file and replacing any fixup lines with pick
- It’s important for my usage to leave the fixup lines in place until I choose to squash things later
Line 15 edits the line for the commit we passed to the original command to put it in edit mode
Line 17 finishes the process off by writing the todo-list back to disk

With all of this in place we can now run a git subcommand that automates the process of rewinding us back to a particular commit to allow us to make changes and then reapply all subsequent commits in order.

Conclusion

There’s a few handy things to take away here around adding your own scripts to git. Knowing that it’s actually not too painful to add quite complex automation means that I’ll really pay attention more to manual tasks that I repeat and look at automating these tasks in future.

Testing tips

27 May 2019

I’ve done a fair amount of data heavy testing recently and here’s some tricks that make things a little easier.

Fixtures
Use #line
Write Comments
Add .make extensions

Fixtures

For this example let’s imagine we have a website selling content subscriptions. We have 3 levels of membership:

enum Membership: Int, Comparable {
    case free
    case basic
    case pro
}

func < (lhs: Membership, rhs: Membership) -> Bool {
    return lhs.rawValue < rhs.rawValue
}

Our User type contains a membership value and a function to determine if a user can access some content:

struct User {
    let membership: Membership

    func canView(content: Content) -> Bool {
        return content.eligibleMembership <= membership  
    }
}

Our content is then marked with an eligibleMembership level:

struct Content {
    let eligibleMembership: Membership
}

To exhaustively test this function we need to write 9 test examples. This is calculated by looking at the inputs to this function.

NB because this is a member function we also count the member variable membership.

User.membership            - has 3 cases (free|basic|pro)
Content.eligibleMembership - has 3 cases (free|basic|pro)

3 cases x 3 cases = 9 test examples

Writing 9 separate tests is going to have a lot of duplication and quite frankly I struggle to name 1 test let alone 9 similar tests that vary only by input.

This is where I would use fixtures and some funky formatting to help readability.

func testCanViewContent() {
    let fixtures: [(Bool, Membership, Membership)] = [
        // expected  |  userMembership  | eligibleMembership
            (true,         .free,              .free),
            (true,         .basic,             .free),
            (true,         .pro,               .free),
            (false,        .free,              .basic),
            (true,         .basic,             .basic),
            (true,         .pro,               .basic),
            (false,        .free,              .pro),
            (false,        .basic,             .pro),
            (true,         .pro,               .pro),
    ]

    for (expected, userMembership, eligibleMembership) in fixtures {
        let result = User(membership: userMembership).canView(content: .init(eligibleMembership: eligibleMembership))
        XCTAssertEqual(expected, result)
    }
}

The basic idea here is to loop over an array of tuples that contain the inputs for all of our examples. In the body of the loop we do the usual arrange/act/assert dance for unit testing.

What I like about the above is:

It’s compact and all cases can be in one test.
Being in a tabular layout makes it easier to read.
I only have to name the test function once.

What’s bad:

Performing assertions in a loop makes it hard to see which case is failing.
- Xcode will just highlight the XCTAssert* line, which is not going to help when we run multiple examples.
It takes a little practice and getting used to.

I think we can do better and solve the first point…

Use #line

To get better feed back #line is your friend. It adds a little more boilerplate to each fixture but my goal for testing is to make it super easy to read the tests and figure out when things go wrong. This is why I’m happy to use an odd tabular format and add a sprinkling of boilerplate.

Here’s the above example with #line added to show how it looks:

func testCanViewContent() {
    let fixtures: [(Bool, Membership, Membership, UInt)] = [
        // expected  |  userMembership  | eligibleMembership  |  line
        (true,         .free,              .free,               #line),
        (true,         .basic,             .free,               #line),
        (true,         .pro,               .free,               #line),
        (false,        .free,              .basic,              #line),
        (true,         .basic,             .basic,              #line),
        (true,         .pro,               .basic,              #line),
        (false,        .free,              .pro,                #line),
        (false,        .basic,             .pro,                #line),
        (true,         .pro,               .pro,                #line),
    ]

    for (expected, userMembership, eligibleMembership, line) in fixtures {
        let result = User(membership: userMembership).canView(content: .init(eligibleMembership: eligibleMembership))
        XCTAssertEqual(expected, result, line: line)
    }
}

The key changes are:

Update every case to take the #line.
Update the call to XCTAssert* to take #line.

With these changes in place Xcode will now highlight the actual fixture that is failing.

Write Comments

Looping back to an earlier point - a goal when writing good tests is that they be easy to read and simple to debug. That’s why it’s really important that you add comments when it makes sense. I often find myself falling into the trap of trying to contort a test function name to reveal all intent but it often ends up in an unreadable mess.

A good comment for a test would explain why some set up is being done. I also find that it’s sometimes really useful to add // Given, // When and // Then comments throughout a test to help delineate the different phases of the function.

Applying these suggestions to the above example might result in this change:

/// Membership levels are ordered `free < basic < pro`.
/// These tests run all permutations to ensure that a user can only
/// view content that is less than or equal to their membership level.
func testCanViewContent() {
    let fixtures: [(Bool, Membership, Membership, UInt)] = [
        // expected  |  userMembership  | eligibleMembership  |  line
        (true,         .free,              .free,               #line),
        (true,         .basic,             .free,               #line),
        (true,         .pro,               .free,               #line),
        (false,        .free,              .basic,              #line),
        (true,         .basic,             .basic,              #line),
        (true,         .pro,               .basic,              #line),
        (false,        .free,              .pro,                #line),
        (false,        .basic,             .pro,                #line),
        (true,         .pro,               .pro,                #line),
    ]

    for (expected, userMembership, eligibleMembership, line) in fixtures {
        // Given
        let user    = User(membership: userMembership)
        let content = Content(eligibleMembership: eligibleMembership)

        // When
        let result = user.canView(content: content)

        // Then
        XCTAssertEqual(expected, result, line: line)
    }
}

Add `.make()` Extensions

If you are writing a lot of tests and you have types that are awkward to create then you’d benefit from creating factory helpers.

Let’s take the following type:

struct User {
    let id: String
    let username: String
    let membership: Membership
}

It only has a couple of properties but it will already be a pain to create lots of users especially if for example we was writing tests that only cared about membership.

We can simplify things by creating an extension only within our test target that provides defaults for all values:

extension User {
    static func make(id: String = "_", username: String = "_", membership: Membership = .free) -> User {
        return .init(id: id, username: username, membership: membership)
    }
}

With this helper in place we can create new users really easily and just provide the values we care about:

greet(user: User.make(username: "Paul"))

// We can take advantage of type inference and drop the type as well
greet(user: .make(username: "Paul"))

Conclusion

Some of these tips are only useful in certain circumstances but I’ve found it really helpful having these techniques in my tool bag.

Older Newer

paul-samuels.com

Git apply-edit - improving `git edit`

Problem outline

git alias

Git edit - improving --fixup

Using --fixup

Problems with --fixup

A solution git-edit

Creating your own git commands

GIT_SEQUENCE_EDITOR

Conclusion

Testing tips

Fixtures

Use #line

Write Comments

Add .make() Extensions

Conclusion

Using `--fixup`

Problems with `--fixup`

A solution `git-edit`

`GIT_SEQUENCE_EDITOR`

Add `.make()` Extensions