«    »

The Role of Quality Assurance

My understanding of the role of QA (quality assurance) in software development has evolved considerably over the course of my career. At first I was unaware of their existence, coding in blissful ignorance. My first interactions with testers who found embarrassing defects in my code gave me the idea that QA was about testing. This still seems to be a common viewpoint in the industry. Often testers belong to the QA department or are sometimes called QA engineers.

I was fortunate, however, to receive the opportunity to reevaluate this viewpoint when I led my first team. I had a tester on my team and appreciated their contribution. Then the company hired a quality assurance manager who talked a lot about process and ISO quality standards, but said very little about testing and did not manage the testers. So this was my first introduction to the idea that testing and QA were different. Further career experiences reinforced the idea the QA was about the process of achieving quality, while testing was one of the activities within this process.

As time progressed and I observed different teams interacting with QA I saw some disturbing trends. QA representatives would audit teams to confirm they were following defined process by looking for evidence in some form of documentation. Teams would oblige by ensuring the documentation was put in place when word spread that the auditor was on their way, irrespective of whether the process had been followed in the first place. Eventually auditors figured this out and started doing random sampling looking back up to an entire year, so teams had to focus more on ensuring the process documentation was updated. There did not, however, seem to be an increased focus on actually performing the activities required by the process. I wrote about this in articles like Drawbacks of Formal Audits and Ten Audit Warning Signs.

There was something else I found disturbing. I joined a company that was certified compliant with ISO 9001, which is a quality management standard. So as a senior developer I was surprised to receive no information at all about expectations concerning my day-to-day activities as a developer. At this point in my career I was aware of the value of activities like version control, code reviews and unit testing in helping achieve good quality software, but apparently whether or not I or anyone else on the team followed these practices had no relevance for ISO 9001 compliance.

Based on these experiences I decided that QA needed to be concerned about the effectiveness of the process first, secondly whether the process was being followed, and only as the lowest level of priority should QA worry about having documented evidence of following the process. A team with significant defects in production or an upset customer has quality issues that QA should be concerned with.

Why is following process important? Alistair Cockburn introduced the concept of habitability for software development methodologies after discovering after surveying many teams that most were not following elaborate processes documented in voluminous tomes. Habitability is simply how easy a process is to follow: the easier it is, the more likely it will be followed. In other words, a process that isn't being followed is actually ineffective in a certain sense, and is evidence that perhaps it should be changed instead of bludgeoning teams into following the process. But as Alistair points out, under certain conditions (e.g. large teams, distributed teams, or mission critical software) more process is required. So the question is whether there is an appropriate amount of process.

I have been comfortable with this concept of QA for a while, but I recently read the book Scalability Rules: 50 Principles for Scaling Web Sites by Martin Abbott and Michael Fisher that caused a further evolution in my thinking. In a chapter titled "Learning from your mistakes" they write the following:

It is QA's job to teach the organization what is happening from a quality perspective and where it is happening such that the engineering [aka development] organization can adapt and learn. QA then becomes a tool to help the organization learn what mistakes it is making repeatedly, where those mistakes lie, and ideally how the organization can keep from making them in the future... QA must work to identify where a growing organization is having recurring problems and create an environment in which those problems are discussed and eliminated.

One could interpret this as talking about continuous improvement, which it is, but it goes far beyond the ISO 9001 level of documenting improvement activity and having a regular quality meeting. The authors talk about QA helping to enable learning and growth, instead of being fixated on processes that may or may not be relevant to the problems being encountered.

In another twist on the common perception of QA, the authors write about QA helping to identify the need for and enforce standards, making no mention of process. While not quite as inspiring to me as the prior revelation, this is still a valuable evolution in the role of QA. A moment's thought makes it clear that standards encompass far more than processes do: a process is essentially a standard for performing a certain set of activities in support of a business objective. There are many other standards relevant to quality such as coding standards, development practice standards, or architectural standards that should also be of concern to QA.

So my current perspective on the role of QA has evolved dramatically over time, and differs significantly from what is often done today within organizations. As an architect I am used to have the desired target state differ from the current state (as per TOGAF): the challenge is in moving teams and organizations towards that ideal state. Hopefully this article will help raise awareness and desire to evolve the role of QA. I would appreciate hearing your perspective and experiences with QA - feel free to leave a comment or send me an email.

If you find this article helpful, please make a donation.

«    »