I considered calling this post, “assertEquals considered harmful” but thought that it might be a bit too harsh. After all, assertEquals is still good for asserting that the primitive result of a calculation is accurate. But I will never again use it for complex objects and I hope to convince as many people as possible that there is a better way.

First off, let’s start with some assumptions:

  1. We want our tests to be clear to the reader and convey what we are trying to test without a lot of extra baggage.
  2. We want our test failures to be clear about what the problem was and not have to spend copious amounts of time figuring out what the problem in the result is.
  3. We want to follow the “One Assert Per Test” TDD rule – or get as close as we can to it.
  4. Implementing the equals method purely for comparisons in testing is a dubious, potentially wrong and harmful, practice.

That last one probably merits a blog all of its own. The problem is that for some objects, domain entity objects as an example, implementing equals and hashCode in terms of the complete state of the object is not usually what you want. Often, for these types of objects, there is an identifier member such that, if 2 objects had the same value for the identifier member, you would consider them to represent the same entity. So you need to think long and hard about why you’re implementing equals and what 2 objects of that type being “equal” really means.

If all that seems a bit abstract for the end of the week after long week of Atlassian Summit and JavaOne, we can consider another case that I ran into recently – the objects that you want to compare are of types from a 3rd party library and the developers didn’t implement equals, hashCode or toString! So, you can’t use assertEquals directly and even if you could any output you get is completely useless!

The pain of assertEquals

So, lets say we’ve got a type called Lightsaber that doesn’t implement equals, hashCode, or toString. How can we compare 2 instances to be sure they’re the same? The plain old JUnit with assertEquals way would be

assertEquals(thisLightsaber.isSingleBladed(), thatLightsaber.isSingleBladed());
assertEquals(thisLightsaber.getColor(), thatLightsaber.getColor());
assertEquals(thisLightsaber.getHilt(), thatLightsaber.getHilt());

That’s all well and good, but there are a few problems. If there is a failure, it’s hard to tell exactly which statement failed. In this trivial example it will be easier because each of the things we’re comparing is of a different type. But if we had multiple assertions on Color or boolean this wouldn’t be so readily apparent. Another problem is that the Hilt object returned from Lightsaber.getHilt is another object, and if it doesn’t implement equals then our test is suddenly going to blow up as we need separate asserts for each of its attributes. So we can do that, and let’s say we even pull these asserts out of our test into a separate assertEquals(Lightsaber, Lightsaber) utility method so we can reuse it all. We’re still left with a problem: the tests only run until the first mismatched field. Instead of showing us all the failures, we only see the first one that is detected which leads to a long test-fix-test-fix-test cycle.

Enter Hamcrest

So, what’s a guy to do? We’ve been using Hamcrest for a while on the SF team. What first attracted me to Hamcrest is its really nice, fluent language for making assertions

assertThat(anakin, is(equalTo(vader));
assertThat("May the asserts be with you", containsString("assert"));
assertThat(weaponsOfGalacticDestruction, hasItems(deathStar, sunCrusher));

but what kept me coming back for more is the way you can define custom Matchers to do the comparisons. Each of those statements above uses the same assertThat method, but a different Matcher is used for the actual comparison. There are all sorts of Matchers built into the core library, but it is trivially easy to implement our own as well.

// factory methods for fluent language
static Matcher<? super Lightsaber> equalTo(Lightsaber lightsaber) {
return new LightsaberIsEqual(lightsaber);
}
static Matcher<? super Hilt> equalTo(Hilt hilt) {
return new HiltIsEqual(hilt);
}
private class LightsaberIsEqual extends BaseMatcher<Lightsaber> {
private final Lightsaber lightsaber;
private final Matcher<? super Hilt> hilt;
LightsaberIsEqual(Lightsaber lightsaber) {
this.lightsaber = lightsaber;
hilt = is(equalTo(lightsaber.getHilt()));
}
public boolean matches(Object o) {
Lightsaber l = (Lightsaber) o;
return lightsaber.isSingleBladed() == l.isSingleBladed()
&& lightsaber.getColor().equals(l.getColor())
&& hilt.matches(l.getHilt());
}
public void describeTo(Description desc) {
desc.appendText("{singleBladed is ")
.appendValue(lightsaber.isSingleBladed())
.appendText(", color is ")
.appendValue(lightsaber.getColor())
.appendText(", hilt ")
.appendDescriptionOf(hilt);
.appendText("}");
}
}
// HiltIsEqual would look much the same, comparing the attributes in the matches() method and 

Now our assertions in our test look like

assertThat(anakinsLightsaber, is(equalTo(lukesFirstLightsaber));

If this were to fail the output would look like

  Expected: is {singleBladed is true, color is GREEN, hilt is {...}}
but: Lightsaber@35324

That’s better, at least it shows us what we should have gotten. But we’re still not seeing what failures there were.

Describing mismatches

If that’s all there was to the story, it wouldn’t be much of a story and wouldn’t be that much more compelling than having an assertEquals(Lightsaber, Lightsaber) method. But the story doesn’t end there, it continues with the release of Hamcrest 1.2. The most important new bit for us is “added mismatch reporting”. Using this we can take care of that second part of the output, with an added benefit.

private class LightsaberIsEqual extends TypeSafeDiagnosingMatcher<Lightsaber> {
private final Matcher<? super Boolean> singleBladed;
private final Matcher<? super Color> color;
private final Matcher<? super Hilt> hilt;
LightsaberIsEqual(Lightsaber lightsaber) {
singleBladed = is(equalTo(lightsaber.isSingleBladed()));
color = is(equalTo(lightsaber.getColor()));
hilt = is(equalTo(lightsaber.getHilt()));
}
public boolean matchesSafely(Lightsaber lightsaber, Description mismatchDescription)
boolean matches = true;
mismatchDescription.appendText("{");
if (!singleBladed.matches(lightsaber.isSingleBladed())) {
reportMismatch("singleBladed", singleBladed, lightsaber.isSingleBladed(), mismatchDescription, matches);
matches = false;
}
if (!color.matches(lightsaber.getColor())) {
reportMismatch("color", color, lightsaber.getColor(), mismatchDescription, matches);
matches = false;
}
if (!hilt.matches(lightsaber.getHilt())) {
reportMismatch("hilt", hilt, lightsaber.getHilt(), mismatchDescription, matches);
matches = false;
}
mismatchDescription.appendText("}");
return matches;
}
public void describeTo(Description desc) {
desc.appendText("{singleBladed ")
.appendDescriptionOf(singleBladed)
.appendText(", color ")
.appendDescriptionOf(color)
.appendText(", hilt ")
.appendDescriptionOf(hilt);
.appendText("}");
}
}
static void reportMismatch(String name, Matcher<?> matcher, Object item, Description mismatchDescription, boolean firstMismatch)
{
if (!firstMismatch)
{
mismatchDescription.appendText(", ");
}
mismatchDescription.appendText(name).appendText(" ");
matcher.describeMismatch(item, mismatchDescription);
}

Now, let’s say I try and do this assertion

assertThat(lukesFirstLightsaber, is(equalTo(maceWindusLightsaber)));

That should clearly fail, and when it does we get the following output (assuming the hilts are the same)

  Expected: is {singleBladed is true, color is PURPLE, hilt is {...}}
but: is {color is GREEN}

But wait, what’s this?!? In the output it only shows us the color of Lukes first lightsaber. Is that a bug? Lets see, they’re both single-bladed, we assumed the hilts were the same, but we know Mace Windu had a purple blade and Lukes had a green one. Why, it’s only showing us the field that didn’t match! How cool is that?!? For this example it would’ve been pretty trivial to determine where the mismatch was, but suppose we didn’t make the assumption that the hilts were the same. Let’s say they’re the same, except for the size. Then the output would have been

  Expected: is {singleBladed is true, color is PURPLE, hilt is {size is LARGE, ...}}
but: is {color is GREEN, hilt is {size is MEDIUM}}

Even with composite objects we can see at a glance what the differences are!

A question you are probably asking is whether this extra effort is really worth it. After all, this is Java and there is a bunch of boiler plate that wouldn’t be necessary if we went the simple route of the assertEquals(Lightsaber, Lightsaber). I can only speak from my personal experience, but doing it this way has vastly improved my ability to write new test and functionality for the OAuth plugins that I have been working on. It was a major pain to go through and write all the matchers for our own OAuth objects and the OAuth.net objects, but it was a write them once and enjoy the benefits for a long time to come. Every new test I write where I see the abbreviated output that tells me the exact fields that were wrong makes me thankful I put the extra effort in because it saves at least 5-10 minutes everytime there is a test failure. And if it helps me, right now, as I’m writing the tests, I feel confident that it will be a tremendous help to whoever might pick up the code after me and be expected to make changes and add new stuff. Because I know from experience what a pain it is to make a change, run the tests and see

  Expected: <BIG GLOB OF OBJECT AND FIELD ATTRIBUTES GENERATED WITH ToStringBuilder>
but was: <OTHER BIG GLOB OF OBJECT AND FIELD ATTRIBUTES WITH MINOR DIFFERENCES GENERATED WITH ToStringBuilder>

What I would typically wind up doing was copying and pasting to a text editor and lining everything up and trying to scan for what changed. But no more! Now my unit tests actually tell me what is wrong!

There’s another huge benefit of creating these equal matchers, you can use these them with Mockito using argThat, try that with assertEquals(Lightsaber, Lightsaber)!

Additionally, after showing this to some of my colleagues here at Atlassian, it was suggested that I should be able to write a Matcher that does a deep comparison of two objects of the same type using reflection. It’s a good idea and is something that I am working on in my 20% time. I don’t expect it to be something that will take away the need to write some custom equal Matchers, but if I can get it to a point where it works in 80% of the cases, then I’ll be very happy as much of the above boiler plate can be eliminated. More on that to come in the future