writing reflective test assertions with swift
Like all quality pieces of software, CycleMaps has lots of tests. The key to writing lots of tests is to make tests easy to write. If your system makes writing tests a chore, then chances are that you won't write them. That's where TDD comes in - by writing your tests first, you make, by definition, your system easier to test.
Like lots of other apps, CycleMaps is primarily an Objective-C application, that's slowly moving to Swift. For the Objective-C testing we use OCHamcrest and OCMockito, which work very well. With the move to Swift we're looking for other tools to make writing tests easier.
tl;dr
MCAssertReflectiveEqual is a testing function for Swift objects, structs, collections and basic types. You can use it to write assertions without writing and maintaining an equals function when you don't need to. Available via CocoaPods or as a single Swift file.
Example
import MCAssertReflectiveEqual
import XCTest
enum StarType {
case death, killer
}
class Star {
private let type: StarType
init(type: StarType) {
self.type = type
}
}
class LookMaNoEqualsFunction : XCTestCase {
func testTheyMayLookSimilarButAreTotallyDifferent() {
let expected = Star(type: .death)
let actual = Star(type: .killer)
XCTAssertEqual(expected, actual) //fails to compile
MCAssertReflectiveEqual(expected, actual) //assertion fails
MCAssertReflectiveEqual(Star(type: .death), Star(type: .death)) //passes
}
}
the problem
Suppose I have a Swift class
class MenuItem {
let title: String
var isEnabled: Bool
let imageName: String
init(title: String, imageName: String) {
self.title = title
self.imageName = imageName
self.isEnabled = true
}
}
This is a simplification of a real class that CycleMaps uses to represent a menu item. The CycleMaps menu is a list of items that varies depending on what you're currently doing in the app. Suppose then I have a MenuProvider that gives me the menu items in order based on some context.
func getMenu() -> [MenuItem]
How do I test that my provider gives me the right menu based on a given context? What I'd really like to write is something like
func testGivenFooCocktailsAreOffered() {
//given some stuff
let expected = [ MenuItem(title: "Martini", imageName: "martiniGlass"),
MenuItem(title: "Bloody Mary", imageName: "celery") ]
XCTAssertEqual(expected, menuProvider.get())
}
Nice and simple, right? Only I can't, because XCTAssertEqual needs to be able to compare the items - and MenuItems are not Equatable. No problem you say, let's make them equatable. Ah but now we have a problem. I have to write a whole bunch of code (the == func) just for the purpose of testing. And this is a trivial example - what if I have a nested hierarchy of objects or structs? Perhaps more importantly, CycleMaps has no need to compare if two MenuItems are the same - why modify the production code? You could add the extension in your test target. But then the class you're testing is not the same class that will run in your production environment and you still have to write a whole bunch of boring equality code. Which of course you'll need to keep up to date every time your class or struct changes.
Reflection to the rescue
Despite Swift's reputation as a rigid (i.e. type-safe) language, it does provide us with some reflective capabilities through the Mirror structure. Mirrors provide us with lots of interesting stuff and is one of the pieces of functionality that's used to power XCode Playgrounds and the debugger. But for the purposes of this post, we're interested in the following:
- subjectType: The type of the subject being reflected
- children: A collection of (String, Any) tuple that describes all elements in the substructure
- displayStyle: An optional enumeration that tells us the kind of item that's the Mirror's subject. Examples include .class, .struct, etc.
- description: A textual description of the mirror
Here's what this all means in the REPL:
> let item = MenuItem(title: "Martini", imageName: "martiniGlass")
> let mirror = Mirror(reflecting: item)
> print(mirror.subjectType)
MenuItem
> print(mirror.children.first)
Optional((Optional("title"), "Martini"))
> print(mirror.displayStyle!)
"class"
> print(mirror.description)
"Mirror for MenuItem"
We have the tools we need - let's build our reflective equality function!
Building MCAssertReflectiveEqual
This is the signature that we are going to implement:
public func MCAssertReflectiveEqual<T>(
_ expected: T,
_ actual: T,
file: StaticString = #file,
line: UInt = #line)
This allows us to compare two items, expected and actual and makes the compiler enforce that they are of the same type, T. #file and #line are compile time directives that will give our function the file and line number that MCAssertReflectiveEqual is called from. We'll use this in our error messages.
First thing we're going to do is dive into a similar looking function where we lose the little type safety that <T> provides. Why we do this will become clearer soon.
public func MCAssertReflectiveEqual<T>(
_ expected: T,
_ actual: T,
file: StaticString = #file,
line: UInt = #line) {
check(expected, actual, file: file, line: line)
}
private func check(_ expected: Any,
_ actual: Any,
file: StaticString = #file,
line: UInt = #line) {
}
Let's do a basic implementation of the check function.
private func check(_ expected: Any,
_ actual: Any,
file: StaticString = #file,
line: UInt = #line) {
let expectedMirror = Mirror(reflecting: expected)
let actualMirror = Mirror(reflecting: actual)
guard expectedMirror.subjectType == actualMirror.subjectType else {
XCTFail("wrong types", file: file, line: line)
return
}
let expectedAsObject = expected as AnyObject
let actualAsObject = actual as AnyObject
if(expectedAsObject === actualAsObject) {
return
}
}
All this does is create a mirror for both expected and actual, and checks that the types are the same. It then casts (or wraps) into an object and checks if the two references are identical (if so, there's no much point in continuing - it's the same thing!). Note that we override the default arguments for file and line in the XCTAssert function. The reason for this is that if it fails, you want to see the file and line that failed, i.e the MCAssertReflectiveEqual assertion in your test.
Now let's deal with the children - remember children are all items in the substructure.
var expectedChildren = expectedMirror.children
var actualChildren = actualMirror.children
guard expectedChildren.count == actualChildren.count else {
XCTFail("different number of children", file: file, line: line)
return
}
First check is simple - they must have the same number of children. Otherwise the substructure is not the same and therefore the expected and actual cannot be the same. So what happens if there's no children?
if(expectedChildren.count == 0) {
if let expectedNsObj = expected as? NSObject, let actualNsObj = actual as? NSObject {
XCTAssertEqual(expectedNsObj, actualNsObj, file: file, line: line)
} else if(expectedMirror.displayStyle == .struct || expectedMirror.displayStyle == .class) {
return
} else if(expectedMirror.displayStyle == .enum) {
XCTAssertEqual(String(describing: expected), String(describing: actual), file: file, line: line)
}
else if(expectedMirror.description.contains("->")) {
print("ignoring closures in \(expected) and \(actual)")
}
else {
XCTFail("cannot compare types", file: file, line: line)
}
}
We basically have five cases:
- Assuming we can cast expected and actual into NSObjects, we'll use their isEqualTo method. This allows us to handle things like Strings, Ints and the like.
- If that doesn't work and it's a struct or a class, then they are equal. Think about it. It's a class, or a struct of the same type (we type checked it above) with no children. Two instances of class Foo { } are equal.
- If it's an enum it becomes trickier. Swift does not currently allow us to dynamically call a function of a pure swift object. There's also no common enum super struct or class that we can use and we cannot get the ordinal (or a raw value, if available), of an enum anyway. Solution is to stick the two enums into Strings and compare those. init(describing) will give us "an unspecified result" ... "supplied by the Swift standard library". Here be dragons. And you could break this if you make your enums CustomStringConvertible and give two enums the same name. But you would not do that, would you?
- Closures. Unfortunately there's no easy way that we've found to compare closures. There's not even a Mirror.DisplayStyle. But What we do know is that there's a -> in the description, so we use that to determine that the item is a closure. We then ignore it. What does it mean to check if two functions are equal? Not sure how to answer that question, given that they are not objects and are not equatable, meaning that we cannot check if they are the same function. Here's the description of a function:
> Mirror(reflecting: print).description
$R0: String = "Mirror for ((Array<Any>, String, String)) -> ()"
- Ae cannot compare anything else that's not covered above yet.
Note that we only check the style of the expected item and don't bother that of the actual one. But remember they must be the same as we type checked them above.
Now let's deal with the children.
while(!expectedChildren.isEmpty) {
let expectedChild = expectedChildren.popFirst()!
let actualChild = actualChildren.popFirst()!
XCTAssertEqual(expectedChild.label, actualChild.label, file: file, line: line)
check(expectedChild.value, actualChild.value, file: file, line: line)
}
We've already checked that the items have the same number of children, so pop each one (notice the force unwrap), and recurse them through the same function.
Dealing with cyclical references
A cyclical reference happens when a child references its parent at any level. For example:
class Loopy {
var loop:Loopy?
}
let a = Loopy()
a.loop = a
What will happen if we try to recursively iterate over a and its children? EXC_BAD_ACCESS will happen, as we blow the stack. In order to tackle this case we're going to have to keep track of what we've recursed from, and bail out if we find a child we've already visited. Let's go back to our original definition of MCAssertReflectiveEqual and modify it.
public func MCAssertReflectiveEqual<T>(
_ expected: T,
_ actual: T,
file: StaticString = #file,
line: UInt = #line) {
var expectedVisited = Set<ObjectIdentifier>()
var actualVisited = Set<ObjectIdentifier>()
check(expected, actual, expectedVisited: &expectedVisited,
actualVisited: &actualVisited, file: file, line: line)
}
private func check(_ expected: Any,
_ actual: Any,
expectedVisited: inout Set<ObjectIdentifier>,
actualVisited: inout Set<ObjectIdentifier>,
file: StaticString = #file,
line: UInt = #line) {
}
We modify our top level function to create two sets to hold items that we have visited, and pass them to the private function. The sets are sets of ObjectIdentifier, which gives us unique Hashable identifiers for class instances. We could have used an array, but searching in the set is an O(1) operation whereas searching in the array is an O(n) one.
Here's how we protect against loops. We're rewriting the else block, when the items being examined have children.
while(!expectedChildren.isEmpty) {
let expectedChild = expectedChildren.popFirst()!
let actualChild = actualChildren.popFirst()!
let canHoldChildrenByReference = expectedMirror.displayStyle == .class
if(canHoldChildrenByReference) {
let expectedIdentifier = ObjectIdentifier(expectedChild.value as AnyObject)
let actualIdentifier = ObjectIdentifier(actualChild.value as AnyObject)
let expectedHasBeenVisited = !expectedVisited.insert(expectedIdentifier).inserted
let actualHasBeenVisited = !actualVisited.insert(actualIdentifier).inserted
if(expectedHasBeenVisited || actualHasBeenVisited) {
if(expectedHasBeenVisited == actualHasBeenVisited) {
return
} else {
XCTFail("looping objects", file: file, line: line)
return
}
}
}
XCTAssertEqual(expectedChild.label, actualChild.label, file: file, line: line)
check(expectedChild.value, actualChild.value, expectedVisited: &expectedVisited,
actualVisited: &actualVisited, file: file, line: line)
if(canHoldChildrenByReference) {
_ = expectedVisited.remove(ObjectIdentifier(expectedChild.value as AnyObject))
_ = actualVisited.remove(ObjectIdentifier(actualChild.value as AnyObject))
}
}
The first thing we do is figure out if we care about loops here. Loops can only happen if we can hold children by reference (i.e. the current type is a class). The next thing we do is see if we've visited the expected child and the actual child before. We do this by checking if there's an identical object in the respective visited sets. If there is in both expected and actual, then they must be equal. Think about it: I'm encountering a cycle at the same point in both expected and actual. Somewhere up the stack I've evaluated the type equality of the already visited object. Therefore they must be equal. If we're only encountering a cycle in only one of expected or visited, then by definition they are not equal. Lastly, after we recurse into the children, we don't need to keep their reference any longer therefore we remove them from the visited collections.
It's a little bit more complicated than that (but only a little)
The actual code is slightly different. We wish to write tests for MCAssertReflectiveEqual, as there's no point to have an assertion function that doesn't work. Hardcoding the use of XCTest assertions makes this difficult, so the actual code allows different functions to be passed in; by default XCTest assertions are used but in the tests we use custom functions that allow us to assert what happened. Finally we construct better assertion failure messages so that you can use this in your code.
We hope you find this useful. If you do, drop us a line at stefanos at zachariadis dot net or even better a pull request.