Page references to xUnit Test Patterns (XTP), especially Chapter 11 ("Test Doubles"), "Four-Phase Test" (pp. 358-361) and Practical Object-Oriented Design in Ruby (POODR), especially Chapter 9 ("Designing Cost-Effective Tests").
Other resources:
The thing we are testing (the SUT)
The thing we are testing is called a "System under test" (SUT) (XTP) or "object under test" (POODR, p. 195, Figure 9.1). We'll use SUT. It is fair to think about other things being under test, such as a class, object, method, or application, but the idea here is that the thing under test is defined by the scope of the test itself. Other objects may be involved, such as a "depended-on component" (DOC).
(POODR, Figure 9.1)
Testing messages received from others
When we receive a message from another object, we want to verify a state change in our own object.
Example: Suppose we have an object that converts fahrenheit to celsius. When we pass in a measurement in fahrenheit, we want to verify that the computation of the celsius value is correct.
require 'minitest/autorun'
class TemperatureConverter
def f_to_c(f)
(f - 32.0) * 5.0 / 9.0
end
end
class TemperatureConverterTest < MiniTest::Test
def setup
@tc = TemperatureConverter.new
end
def test_f_to_c
assert_in_delta 0.0, @tc.f_to_c(32), 0.01
end
endThis is very easy. The SUT does not depend on other classes; the state change can be verified directly.
Testing messages received from others -- When to use a stub
Now let's say that we have a converter object that is capable of a variety of conversions. The way this is going to work is that we are going to pass in the type of conversion we want, along with a value to be converted, and we want to get the right result.
It will look something like this:
Converter.new(FToCConverter).convert(32)
Now our Converter class depends on a collaborator, FToCConverter. XTP calls this an "indirect input" (p. 125), and the name it gives to this kind of collaborator is a DOC -- a "depended-on component." We could test Converter with a specific Collaborator (FToCConverter) or we can try to test the SUT in isolation. If we can test the SUT in isolation, then there will be fewer dependencies on other objects, which will make our test less brittle.
When we test the converter, we are not attempting to establish whether the conversion is correct; instead, we want to verify that it can delegate the 32 to the specific converter and return a value. For the verification of the specific converter, we will write separate tests for that. Additionally, we may be writing the main converter first. We may not even know the range of specific converters we are going to want, or have one in hand.
Stubbing
A stub is an implementation that returns a canned answer (POODR, p. 210).
NOTE: We stub on the DOC, not on the SUT. For some guidance on this, see https://robots.thoughtbot.com/don-t-stub-the-system-under-test.
If we create a stub manually, we will want an instance of FToCConverter's convert method to return a value that we can verify. It might look like this:
require 'minitest/autorun'
class FToCConverter
def convert(value)
500
end
end
class Converter
def initialize(specific_converter_class)
@specific_converter = specific_converter_class.new
end
def convert(value)
@specific_converter.convert(value)
end
end
class ConverterTest < MiniTest::Test
def setup
@c = Converter.new(FToCConverter)
end
def test_convert
assert_in_delta 500, @c.convert(32), 0.01
end
endNOTE: In the real world, this is not quite how it's done. Why? Because in this test, we don't care about
FToCConverter. All we care about is making available to the test an object that exposes a convert
method that returns a specific value, so that we can validate that the main convert method on
Converter leverages it. In short, we want to write the least amount of code to see that the plumbing
is working. In MiniTest, it might look like this:
require 'minitest/autorun'
class FToCConverter
def convert(value)
500
end
end
class Converter
def initialize(specific_converter)
@specific_converter = specific_converter
end
def convert(value)
@specific_converter.convert(value)
end
end
class ConverterTest < MiniTest::Test
def specific_converter
@specific_converter ||= FToCConverter.new
end
def setup
@c = Converter.new(specific_converter)
end
def test_convert
specific_converter.stub :convert, 400 do
assert_in_delta 400, @c.convert(32), 0.01
end
end
endNotice that here our implementation of FToCConverter#convert returns 500. But our stub of
this method returns 400: and we verify against that. What this means is that we can check the plumbing
of the delegation to the specific converter even if it's wrong; our stubbed test doesn't depend on
what the actual class does at all: our test is completely independent. Additionally, we have moved the
key verification value closer to the assertion, which is easier to read.
For here on out, we will use a different syntax in MiniTest. Here's the same test but using spec-style syntax using keywords such as "describe" and "expect."
require 'minitest/autorun'
class FToCConverter
def convert(value)
500
end
end
class Converter
def initialize(specific_converter)
@specific_converter = specific_converter
end
def convert(value)
@specific_converter.convert(value)
end
end
describe Converter do
let(:specific_converter) { FToCConverter.new }
subject { Converter.new(specific_converter) }
it "can delegate to a specific converter" do
specific_converter.stub :convert, 400 do
expect subject.convert(32).must_be_within_epsilon 400, 0.01
end
end
endlet and subject provide for dynamically creating methods that return what's defined in the blocks.
With let, the name of the method comes from the symbol you pass in. For subject, the method
is called subject. They are both lazy. So if you never use specific_converter the block { FToCConverter.new } will
never run. Also, because they are lazy, you can reverse the order. Maybe people like to have the
subject of the test at the top of the spec. I'll do that below.
Stubbing in MiniTest requires the the stubbed method actually exist in the instance being stubbed. Other
testing frameworks are more lenient in this respect, which can result in more concise tests -- but at
the expense of having test doubles that are too fake and don't match up to your real collaborators. For
example, you might stub a method convert but over time your collaborators change that method name
to transform: Your tests would continue to pass because you've stubbed a method that doesn't exist
on the real objects.
The Four-Phase Test
It is conventional to think of tests as having four phases:
- Setup
- Exercise
- Verify
- Teardown
Typically teardown (releasing resources) is done for you by the framework. Here's our spec of Converter
with comments showing the phases:
describe Converter do
let(:specific_converter) { FToCConverter.new } # setup
subject { Converter.new(specific_converter) } # setup
it "can delegate to a specific converter" do
specific_converter.stub :convert, 400 do
converted_value = subject.convert(32) # exercise
expect converted_value.must_be_within_epsilon 400, 0.01 # verify
end
end
# teardown
endTesting messages sent to others -- when to mock
When we send a message to another object that results in a side effect, we want to verify the side effect.
In other words, we want to prove that a promised behavior change in a collaborator has been triggered.
Example:
Our Converter provides a means for there to be logging of the conversion.
Let's make a few changes to our design of Converter. First off, let's allow that it's easier to
provide configuration parameters via a hash. We'll also provide sensible defaults -- a specific converter
that does nothing and nil for the logger. We won't trigger a logger if no logger is set.
class Converter
class PassThroughConverter
def convert(value)
value
end
end
attr_reader :converter, :logger
def initialize(args = {})
@converter = args[:converter] || PassThroughConverter.new
@logger = args[:logger]
end
def convert(value)
converted_value = converter.convert(value)
log(value, converted_value)
converted_value
end
private
def log(value, converted_value)
logger.log(value, converted_value) if logger
end
endNow, what do we want to verify? We do not want to verify that the log method on Converter gets
called with value and converted_value -- what we want to know is whether the collaborator is
sent the right message. In this case, we want to know if a logger instance would be sent
the message log with the right values. At this point, we don't even have a logger class. We just
know that it is going to expose a method log and expect that the method call will pass the
value and its conversion.
To make this happen, we want to create a Mock. Notice that we are still using the stubbed converter.
But now we write logger.expect to set up our expectations for what will happen on the collaborating
object; and then we verify it afterward.
It is critical to understand that we are not mocking an object or a class; we are just verifying
that the delegate is sent the log method with the right parameters. What this means is that we are
verifying a "role" -- Just one aspect of the outgoing messages from the SUT.
I've annotated this with the four phases.
describe Converter, "delegation to logger" do
subject { Converter.new(logger: logger) } # setup
let(:logger) { Minitest::Mock.new } # setup
let(:converter) { subject.converter } # setup
it "logs the value and the converted value" do
converter.stub :convert, 400 do
logger.expect(:log, nil, [32, 400]) # verify
subject.convert(32) # exercise
end
logger.verify # verify
end
endOther topics to add
- Metz's method for verifying that an object conforms to an interface
- RSpec's "behaves like" pattern
- Fixtures
Terms
- SUT - System under test
- DOC - Depended-on component
- "indirect input" - This is data that gets into the SUT from a DOC. I.e., we're not calling a method with parameters; inputs are getting into the SUT via some DOC. (126)
- "indirect output" - We want to verify at an "observation point" that calls to the DOC are happening correctly (127).
- "Stubbing" for Indirect Input - When the SUT makes calls to the DOC, it may take data from the DOC. This data would be an "indirect input." We want to simulate these indirect inputs. Why? Because the DOC may be unpredictable or unavailable. A thing that stands in for the DOC so as to provide indirect inputs to the SUT is called a stub. The stub receives the calls and returns pre-configured responses. We want to "install a Test Stub in place of the DOC" (129). Want to provide indirect inputs? We say that install the Test Stub to act as a "control point" (135). We call it a "control point" because we are trying to force the SUT down some path (524).
- "Test Spies" or "Mocking" for Indirect Output - By "indirect output," we mean the calls the SUT makes to DOCs. Example: the SUT makes calls to a logger. We want to ensure that the DOC is getting called properly.
- Procedural Behavior Verification. We want to capture the calls to the DOC during SUT execution and see what happens. This means installing a Test Spy. It receives the calls and records them; then afterwards we make assertions on what is recorded in the Spy. What to check indirect outputs? We say that that happens at an "observation point" (e.g., 137).
- Expected Behavior. We install a Mock Object, and say in advance what we expect. If the Mock doesn't get what we expect, it fails the test.