In Part 1 of this series I described the characteristics that make up a good Automated Test. Here in Part 2 of this series I will explore all of the benefits you will enjoy by creating those good tests and why the time spent on making good tests is a no-brainer investment.

What are the upsides of Automated ?

If you make the commitment to have all of your tests meet those good quality characteristics from Part 1, what do you get in return?

fearless

Refactoring in simple terms is just the changing of existing to improve the overall quality of that . If you have a working, tested base, then there is always a level of risk when changes are made, whether they are additions, deletions or just changes. Changing one piece of may cause a problem in another piece of and introduce a bug there, even though the change was innocuous. So think of your suite of tests as your safety net that allow you to make changes and be alerted quickly if there was a problem introduced. This means a new feature or bug fix can be implemented quickly and with confidence and that can be given to the QA team as quickly as possible.

shorter implementation time

Having a suite of tests as your constant sanity check and safety net allows developers to move forward with greater velocity. They are not encumbered by the need to verify they changes they are making on a minute-by-minute basis. They are confident in the fact their code is being monitored and inspected automatically by the tests they and their fellow developers are consistently adding to the .

In the end this means that in an style project a developer can actually commit to and complete more within a single iteration and in a traditional project it means that the total implementation will be shortened and QA will get their hands on the code base sooner.

shorter QA cycles

If the overall quality of code coming out of the development team is higher, then obviously the QA team is going to find fewer issues. Code that is subjected to a battery of automated tests will inevitably be of a higher quality than code that is not (this is only limited by the quality and quantity of tests being written).

Traditional QA cycles consist of a large amount of overhead when compared with the actual forward progress made towards releasing the . There is a lot of context switching by both the development and QA , there is overhead in moving builds into QA environments, there is overhead in determining what really is or isn’t a bug, there is overhead in logging, tracking and closing bugs and there is a communication overhead incurred between the development and QA if they are not tightly integrated (which unfortunately is the common model today).

A traditional end-of-project QA cycle really is very inefficient, so delivering higher quality code to the QA team in the first place will reduce the amount of time spent working in this inefficient manner.

no QA cycle

Of course the ultimate zen state would be to eliminate the QA cycle all together. And before the QA folks have an aneurysm, notice I said “eliminate the QA cycle”, not “eliminate the QA team”. I definitely believe there is a role for a QA specialists within a software team, however I believe their role should be a lot different than the role they traditionally perform, but that is a whole topic on its own for another post.

If your test suite is so comprehensive, what is left to do in a QA cycle? With that test suite backing you up, the decision to release an update to your code is based entirely on whether or not all the tests pass, that’s it (plus your Product Manager blessing the feature set I presume). This of course requires a level of confidence in your tests that very few teams could say they have these days. However, there are many examples of companies starting to eliminate the QA cycle altogether and even going a step further to automatically migrate changes to production environments if the test suites run successfully – Google the terms “continuous delivery” and “continuous deployment” for more details.

There are some testing strategies usually referred to with names like “free form” or “ad hoc” that do have value, but are hard to automate, so would still need to be done by hand if you want that kind of testing done. I would argue however that while that style of testing does have value, the value is so insignificant in comparison to the value that you can achieve with comprehensive , that holding up a release to do that kind of testing is folly. If you do want to do that kind of testing, why not just do it on the released version and capture any minor or obscure issues to be addressed in the next release.

shorter total development time

Many developers and project managers often underestimate the amount of time needed for the QA at the end of a traditional project. For example, QA cycles estimated at 1 or 2 weeks can easily become 1 or 2 months long or even longer when dealing with a code base of low quality.

It is important to remember that the total implementation time for a greenfield project is all of the hours spent by developers from the time the first line of code is written until the release goes live to customers. Until the customers have and are able to use it, the project has achieved very little, possibly nothing. Getting the code into the QA team’s hands for the first time is really only a minor achievement if the code is of a low quality.

In a traditional project the work by developers is broken up in to a large chunk up front as the bulk of the functionality is developed, followed by smaller chunks as each QA build is rejected and the developers fix those bugs, until the product reaches an acceptable level of quality to allow it to be release (which is rarely a bug free release, but some compromised lower level of quality). Getting the release into QA’s hands for the first time is really only a minor achievement, although it is often celebrated and lauded wildly.

For existing projects already with a comprehensive suite of tests, that are being enhanced, the benefit is even greater. This benefit comes from the same place as the Fearless Refactoring as stated above. Adding an enhancement is basically a large scale Refactoring, so if you know everything is working before you start, and everything is still working when you are done, you can confidently make that first QA release.

eliminate bug regressions

This is a classic adage of the automated testing advocates – every time a bug is reported, the first step is to write a test that exposes the bug, only once you have that test in place do you start to fix the bug. This does two things, firstly it allows the bug to be verified, and secondly once the test is in place and the fix is in place it is going to be very difficult for that bug to slip back into the product in a future release.

Imagine if your QA team who is finding or triaging these bugs were skilled enough to write the test that exposes the bug? You just eliminated the classic QA vs developer impasse where a bug is reported by QA in one environment but the development team can’t replicate it, which obviously wastes a huge amount of time.

higher quality end product

When low level logic bugs (like an off-by-one problem when working with an array for example) are eliminated early in the development cycle, it frees both the developer and QA teams up to worry about bigger, higher level issues. Developers can worry about architectural concerns, like making sure code has all of the illities in place. The QA team can look for more complex issues, like dealing with issues that only appear under certain environmental conditions.

In the end, the overall quality of the software cannot help but improve when a disciplined approach to automated testing is followed. Whether you reach the holy grail of a “bug free” release is of course up to you.

Conclusion

It should now be easy to see why investing in an automated testing strategy will bring an overwhelmingly positive ROI for your development team. Automated testing will allow your development team to become a reliable and productive part of your organization, that can be counted upon to meet deadlines on a regular basis. But don’t be fooled, even with all of this potential ROI upside, you will still find many naysayers that will try to block your efforts. And that is what I discuss in Part 3 of this series.