Testing Execution Order

August 7, 2012

I continue to work on tightening my tests and accounting for all behavior on which the success of my code depends. One concept mentioned by my mentor today is that of order-of-execution. Currently, many of my tests are actually brittle in respect to what exactly is being tested. The appropriate method calls may be made, but not necessarily in the correct order.

class MyClass
  def run
    do_first
    do_second
  end

  def do_first
    ...
  end

  def do_second
    ...
  end
end

describe MyClass do
  it "#run calls methods #foo and #bar" do
    obj = MyClass.new
    obj.should_receive(:do_first)
    obj.should_receive(:do_second)
    obj.run
  end
end

This test should pass, confirming that methods #do_first and #do_second are indeed both called by the #run method. However, as can likely be inferred by the example names given, there is a requirement for execution order that is not being tested. We reveal the problem by swapping the call order inside the #run method.

...
  def run
    do_second
    do_first
  end
...

We’ve changed the call order, but the tests still pass! If method #do_second is dependent upon the effect #do_first, the behavior will have changed without our test warning us about it. If we would prefer to stick with this type of test, we need to account for the execution order requirement.

###Ordered Method Expectations As concisely illustrated in this post by Tim Riley, RSpec provides a method for expectations to fail if messages to objects are received out of order.

...
    obj = MyClass.new
    obj.should_receive(:do_first).ordered
    obj.should_receive(:do_second).ordered
    obj.run
...

Making this addition to the expectations in this test ensures that the methods are called in the correct order.

###Other Thoughts There are many different ways to test the ideas implemented in code. Some are more effective than others. Though, there doesn’t appear to be magic bullet that slims down tests while both increasing coverage and decreasing brittleness.

< A Few More Things Learned About Testing