Why CollectionAssert.AreEqual fails even when both lists contain the same items

Last time you did this, it worked flawlessly. The test you made verified that an item was added and then returned correctly:

  • Retrieve all items into a “before” list.
  • Add an item.
  • Retrieve all items into an “after” list.
  • Manually add the item to the “before” list.
  • Call CollectionAssert.AreEqual to compare the “before plus new” and the “after” lists.

The test passed. Just as it should.

Now you have almost exactly the same test. Only thing that has changed are the items in the lists that you are testing.

And now the test fails.

Why?

Psychic debugging isn’t my strongest skill and any number of a whole bunch of things could be the matter. But I’ll give it a stab, so let’s have a look at the test method.

    public void AddingMountain_Should_StoreIt_And_ReturnIt()
    {
        // Arrange
        List<Mountain> beforePlusNew = new Earth().GetAllMountains();
        Mountain mountain = new Mountain(8848, "Everest");
        
        // Act
        new Earth().AddMountain(mountain);
        List<Mountain> after = new Earth().GetAllMountains();

        // Assert
        beforePlusNew.Add(mountain);
        CollectionAssert.AreEqual(beforePlusNew, after);
    }

Oh, right. Obvious. Who in their right mind would create multiple Earths?

Seriously though, instantiating Earth three times means that the items in the beforePlusNew and after lists may seem but not necessarily be the same.

There is a difference between reference and value equality.

That difference is rather important when you are using CollectionAssert.AreEqual to compare items.

According to MSDN CollectionAssert.AreEqual:

Verifies that specified collections are equal. Two collections are equal if they have the same elements in the same order and quantity. Elements are equal if their values are equal, not if they refer to the same object.

Reading that you would expect that CollectionAssert.AreEqual goes through a loop of sorts, checking the values of items at the same index in expected and actual .

You’d be right.
Well… half right.

A loop is used, but CollectionAssert.AreEqual does not check values itself. It defers to Object.Equals.

Which becomes clear when you read the documentation for the CollectionAssert.AreEqual(ICollection expected, ICollection actual) overload:

Two collections are equal if they have the same elements in the same order and quantity. Elements are equal if their values are equal, not if they refer to the same object. The values of elements are compared using [Equals](http://msdn.microsoft.com/en-us/library/system.object.equals.aspx) by default.

Ah!
This is where the monkey comes out of the sleeve. [1]
I hope.

If Equals determines whether two elements are considered equal, then the test passing in one case and failing in the one you are working on now, must have something to do with the way how Equals compares the elements in your lists.

MSDN has a lot to say on Object.Equals. And it holds the key to the apparent inexplicable behavior that has you pulling your hair out.

I suggest you take some time out at some stage and read it. It does a nice job of explaining the difference between reference and value equality. For now the important thing to notice is that Equals treats reference and value types differently.

According to the docs (paraphrased):

  • Value types are equal if they are of the same type and their public and private fields have the same value.
  • Reference types are equal when they are the same object. For reference types a call to Equals is equivalent to a call to ReferenceEquals. Reference equality means that the object variables refer to the same object.

Well now, isn’t that a bummer. MSDN is lying!

“Elements are equal if their values are equal, not if they refer to the same object.” isn’t true at all! That is only true for value types! What use is that!

Calm down. Equivalent != equal.

Where ReferenceEquals cannot be overridden, Equals can!

In order to get your test to work for lists of mountains (fairly safe bet that this is a class rather than a struct), you could have Mountain override the Equals method and specify exactly when one mountain instance is equal to another mountain instance.

Still, I wouldn’t. Not in this case.

Overriding Equals has a couple of nice little pitfalls.[2] Besides, there is a much simpler way.

The simplest solution is simply to not use CollectionAssert.AreEqual(ICollection expected, ICollection actual).

Huh?

Yes, really. And no, I don’t want you to do it all by hand.

Just use CollectionAssert.AreEqual(ICollection expected, ICollection actual, IComparer comparer).

That overload allows you to tailor the comparison by CollectionAssert.AreEqual to the exact needs of (each of) your test(s).

To use that overload, all you need to do is to add a class that can compare mountains:

    private class MountainComparer : Comparer<Mountain>
    {
        public override int Compare(Mountain x, Mountain y)
        {
            // compare the two mountains
            // for the purpose of this tests they are considered equal when their identifiers (names) match
            return x.Name.CompareTo(y.Name);
        }
    }

In the above example I made it a private class of the test class. You could of course also make it available to all test classes in your test project. Up to you. When you do, I would definitely give it a more descriptive name.

Using a comparer, the call in the test method with which we started this story, would become:

    CollectionAssert.AreEqual(beforePlusNew, after, new MountainComparer());

That’s it. Now you never have to wonder again why CollectionAssert.AreEqual fails when it should pass. Enjoy!

Hitting any roadblocks in getting your code under test? Please do feel free to let me know! I’d love hearing from you by email or in the comments below. I read everything and will try to help where and as best I can.

Notes

[1] The monkey comes out of the sleeve: dutch saying.

[2] Like having to override GetHashCode as well, or that you shouldn’t really do it for mutable types, and that if you do it you should really override the == operator. Read the docs on the Equals method for more information.



6 comments on “Why CollectionAssert.AreEqual fails even when both lists contain the same items
  1. Unlike the [1] footnote which has a reference to it, the [2] footnote does not. I assume you mean to reference from “Overriding Equals has a couple of nice little pitfalls.” (;

  2. Jesse Z says:

    Thanks for taking the time to write this up Marjan, it is a nice simple resolution

  3. Jeremy says:

    In Overriding the Compare method,
    Here they have compared only the mountain.name.
    My question is:
    How to compare all the properties of Mountain when overriding the compare method?

  4. Marjan Venema says:

    The purpose of the Compare method is to indicate whether x should come before y, y should come before x, or whether they can hold the same position.

    To decide you can of course check multiple properties in the result of the Compare method. Start with the “most significant”. When that says that one should should come before the other, you return with the value returned by comparing that property. If the comparison of that property says both x and y get the “same position” then you continue with the next property. Keep doing that until you run out of properties to compare.

    For example:

        public override int Compare(Mountain x, Mountain y)
            {
                int Comparison = x.Name.CompareTo(y.Name);
                if (Comparison <> 0) return Comparison;
           
                Comparison = x.SomeOtherProperty.CompareTo(y.SomeOtherProperty);
                if (Comparison <> 0) return Comparison;
    
                // same thing for other properties you want to compare
           
                return x.LastProperty.CompareTo(y.LastProperty);
            }
    

    But if you really want to compare all the properties then you probably would be better of overriding the Equals method for the Mountain class. Just be aware of the pitfalls involved in doing that. See the doc on the Equals method for more information.

  5. Sergiy says:

    Thanx man.
    You helped me a lot.

    By the way, it should be “!=” instead of “<>”. Probably from some other language.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Show Buttons
Hide Buttons