Larder

What's in your Larder: iOS testing frameworks

One of the main things I've focused on in 2018 has been adding tests to my iOS apps. I've fumbled my way from having no tests at all in a fairly big, one-person project, to refactoring code to be testable, and even using TDD to develop new features. Along the way I've tried many testing frameworks. Here are some that I've had the most experience with, and why I do or don't like them.

Quick

Quick is a popular BDD (behaviour-driven development) testing framework. It helps you structure your tests in a way that I found appealing at first, but ultimately moved away from.

Because Quick wraps XCTest under the hood, it sometimes behaves in unexpected ways, and I had trouble writing my Quick tests in a way that let me share code and use helper functions between different tests. I'm sure this is possible—it just wasn't simple or obvious enough for me to stick with Quick.

I also had a lot of trouble getting Quick to work with EarlGrey or Sencha (mentioned below), and I think that's because they both extend XCTestCase so I was essentially putting one test case inside another.

Nimble

Nimble is made by the same team as Quick, and I'm still using it in most of my tests, even without Quick. Nimble is a matching framework to help you write more readable expectations. Though this might seem like overkill, it makes writing tests much nicer, and I found the syntax easy to pick up.

Here's an example of a simple XCTest assertion:

XCTAssertEqual(1 + 1, 2, "expected one plus one to equal two")

And here are some examples of how you might write assertions with Nimble, taken from the README:

expect(seagull.squawk).to(equal("Squee!"))
expect(seagull.squawk).toNot(equal("Oh, hello there!"))
expect(classObject).to(beAKindOf(SomeProtocol.self))
expect(actual) == expected
expect(actual).to(beGreaterThan(expected))
expect(actual) >= expected
expect { try somethingThatThrows() }.to(throwError())
expect(actual).to(beEmpty())

Anyway, you get the idea. There are lots, lots more supported assertion types in Nimble. Check the project's README for lots of examples.

Nimble Snapshots

I try to avoid snapshot tests, as I've found them to be the most flaky variety of tests, especially when dealing with animated UIs. But when I do write snapshot tests, I use Nimble_Snapshots by Ash Furrow. This library is a wrapper around FBSnapshotTestCase formerly by Facebook and now maintained by Uber. It lets you use Nimble assertions to create and compare snapshots, so your assertion can look something like this:

let view = ... // some view you want to test
expect(view).to(haveValidSnapshot())

If you're already using Quick and/or Nimble, it's really handy to be able to use snapshots without changing the style of your tests.

EarlGrey

EarlGrey is a functional UI testing framework from Google. It lets you write and run UI tests like unit tests, and waits for things like animations to complete before checking for UI elements. EarlGrey is handy for interacting with your app as a user would, and testing that the UI looks and behaves as expected. You can scroll, swipe, and tap, as well as checking the visibility of labels, buttons, and other UI elements. The library supports both Objective-C and Swift, which is handy for mixed projects.

Here are some examples of how you might use EarlGrey to interact with your app during tests:

// perform a tap
[[EarlGrey selectElementWithMatcher:grey_accessibilityID(@"ClickMe")]
    performAction:grey_tap()];
// assert something is visible
[[EarlGrey selectElementWithMatcher:grey_accessibilityID(@"ClickMe")]
    assertWithMatcher:grey_sufficientlyVisible()];
// scroll down, then tap
[[[EarlGrey selectElementWithMatcher:aButtonMatcher]
    usingSearchAction:grey_scrollInDirection(kGREYDirectionDown, 50)
 onElementWithMatcher:aScrollViewMatcher]
    performAction:grey_tap()];

It's surprisingly fun to watch these tests run in the simulator, as it feels like watching someone use your app very quickly.

There's also EarlGreySnapshots in case you want a snapshot testing wrapper to use alongside EarlGrey tests. It lets you write assertions like so:

EarlGrey.select(elementWithMatcher: grey_kindOfClass(AViewToSnapshotClass.self))
    .assert(grey_verifySnapshot())

Sencha

Sencha is a wrapper around EarlGrey that gives it a nicer syntax to work with. It also offers handy convenience methods for opening view controllers and embedding them in navigation controllers, which I use a lot when setting up tests.

Compared to EarlGrey, using Sencha is much nicer. Here are some examples of what it can do:

tap(.accessibilityID("AnythingTappableID"))
tap(.text("ButtonTitle"))
type("Username", inElementWith: .accessibilityID("UsernameTextFieldID"))
tapKeyboardReturnKey()
assertNotVisible(.text("EmptyStateText"))
assertVisible(.class(UIActivityIndicator.class))
assert(tableViewWith: .accessibilityID("TableViewID"), hasRowCount: 30)
assertSwitchIsOff(.accessibilityID("SwitchID"))

I use Sencha in all my UI testing these days and find it really handy, but it is limited, and I often have to use EarlGrey for things that aren't wrapped by Sencha, such as swiping.

---

Finally, here are some other testing frameworks I haven't tried that you might want to check out:

• KIF (Keep It Functional—functional UI testing)
• Cedar (BDD-style testing in Obj-C)
• Specta (TDD/BDD framework in Obj-C)
• Snap.swift (Snapshot testing alternative to FBSnapshotTestCase)

Happy testing!