Submitted by douglas.w.needham on

When it comes to computer programmers and even computer aficionados, there are a number of things which are subject to debate, and at times, these can almost approach a religious holy war given the right individuals. Examples of some of these include "PC" vs. "Mac", "Windows" vs. "*NIX", "vi" vs. "emacs" and more. So it is of no surprise that even within testing, and specifically unit testing, there are some areas were competing  ideas can cause similar reactions. I am going to talk about several of these, as I write unit tests for this code, which ingests/digests log records and adds them to a database.  (I did not do test driven development, TDD, because I needed the code sooner than I could do if I refreshed my PyTest knowledge).

What is a unit and where to draw the line on mocking

One of the problems is what exactly is the definition of the unit, sometimes referred to as the system under test (SUT), and sometimes class under test or code under test (CUT), especially when it comes to OOP. One of the goals of "good" unit tests is that they are quick and easy to write, understand and execute. As a result, I have worked at places which have defined the unit to be the smallest possible callable block, and insisted on mocking pretty  much all of the calls it made, while other places were just of the mindset of "mock to avoid calling outside of the class". And part of this also depends on the language and the mocking framework itself. Let's look at the advantages and disadvantages of mocking all calls.

Advantages:

  • Unit tests are small and "simple", if you ignore the potential complexity of the mocks.
  • There is quick isolation of where something is broken.

Disadvantages:

  • Some situations can still lead to a relatively "simple" method having lots of mocks, such as where the method is one large switch/case statement which cannot be split into multiple methods.
  • Testing methods in total isolation does not reveal dead methods which are not called by any other code.
  • Depending on the language and framework, it may not be able to mock calls to some system functions, such as those dealing with strings. At that point, do you write a wrapper function which you can mock but not directly test, or do you just go ahead and call it without mocking?

  • Depending on the framework and the task, mocking may be less than straight forward to setup, even in situations where mocking a call is critical, such as when a database call is being made. An example of this would be setting up the mocking for this relatively code snippet:

    try:
    if self.dbconn is not None:
        with self.dbconn.cursor() as cursor:
            cursor.execute(query, parameters)
            self.dbconn.commit()
except Exception as e:
	self.dbconn.rollback()
	raise Exception("Error saving IPv6 record") from e

    Even with techniques like dependency injection, this simple block can be quite interesting to test with mocking.

  • Mocking produces multiple places where test code must be updated if the behavior of a method is changed. This is trading complexity at the individual test level for complexity at the overall testing level. It also leads to...
  • Mocking can result in what I call "flying pigs" or "flying boulders", where the code works great in isolation (a pig works as a pig, wings work as wings, but combined, they don't work).

This is no doubt why Osherove used the word "art" in the title of his book "the art of UNIT TESTING". And it is why I tend to have a soft line on where I may or may not mock calls to other methods in the class, but will in true unit tests always mock out calls to certain calls outside the class, such as those which return mutating data such as the date, or a database call.
 

DRY vs. multiple assertions per test

DRY, or the principle of not repeating yourself in code, is another important idea which I can generally get behind, but again can be taken to excess. And, when dealing with unit tests, there is the competing idea of a single assertion per test, and writing separate tests to see if function/method A does tasks X, Y and Z (like removing a value from an array and places it in another location). While it would be nice if it did tasks X and Z, but did not do task Y correctly from the start, I assert that when you start to work on the test, it is simple enough to comment out the assertion to verify task Y and see if task Z  also works or fails, since unit tests are supposed to be quick and easy to execute, especially if you can execute individual tests in isolation.

 The hornet's nest in the thicket

While a number of us always strive to do test driven development, sometimes life is just not that easy. Sometimes, we need to turn out a small utility quickly, and for some reason, writing unit tests would delay things excessively, so we just accept the technical debt. And sometimes, we inherit a brownfield project which has little if any unit tests. If we can, we should always strive to leave things a bit cleaner/greener, and put code we work on under test. But then, I have worked on refactoring methods where the code for the method, without comments, is 1000-1500 lines or more of a massive switch/case statement with the code embedded in each case calling out to a method to communicate with a piece of network equipment. This is like mowing a long untended yard/field and hitting a hornet's nest. Sometimes, you just have to write a test, get one branch of code under test, then refactor the code to put a block of code into a method, run the same test unchanged, and then refactor the test itself. Unfortunately, this takes time, and if the parts you don't fully understand but have to mock out from the start (such as those API calls) have some odd behaviors at times... you can get stung.

Conclusion

In summary, it is important to remember that nothing is a question of black and white, but greys and even colours. Even if your unit tests end up being more integration tests than unit tests, so long as they are quick to execute and reproducible, that is, in my book, the first consideration, so long as you remember there is always room for improvement. That way, you can at least find regressions when making changes, or exercise code which is only exercised when errors occur.

Categories
Unit Testing
TDD