• Highly variable quality - some of it very poor - in delivered work.
• Budget and schedule overruns due to inaccurate estimates and lack of scope control.
• Communication breakdowns.
• Insufficient monitoring and risk mitigation by management.

These issues are all symptoms of the underlying problem of failing to adjust development practices to scale up from one developer to a team. One developer working alone on minor enhancements to an application can typically get by on their own ability, without explicit use of any development practices. The enhancements are often simple enough to fall within the developer’s zone of competence so issues are infrequent. Tackling a larger enhancement with a small team of three or more developers introduces the following scaling factors:

• Higher complexity of the changes (in addition to the larger magnitude of change).
• Higher likelihood of requirement changes, additions, and refinements by the business.
• Communication and coordination among the developers.
• Dealing with variations in practices or abilities between the developers.
• Ramp up needed for new developers to become familiar with the application and the business domain.
• Less flexibility to recover from issues due to larger budgets, longer schedules, more people, and more management visibility.

So what can these teams do to successfully scale? One idea that I often hear mentioned is to turn the work into a formal project and assign a project manager. However, this is seldom a viable option due to considerations like the following:

• Starting a formal project within the organization is a lengthy endeavor that the business wants to avoid since they want their changes made as soon as possible.
• Adding a full-time project manager to a team of three developers represents too much overhead that the business (or I.T., depending on the organization) does not want to pay for.
• The I.T. organization does not have sufficiently skilled project managers to staff these 'mini' projects full-time. (I have been burned by overly-junior project managers making things worse, rather than better.)

So what options are available to teams needing to scale up? Based on some recent experiences, I believe that the Scrum development method has a number of ways it can help. And I am not even talking about fully adopting Scrum (although that is an option).

I will start with the role of ScrumMaster which I have explored in a recent article titled The Mindset of a ScrumMaster. The role of the ScrumMaster is to help grow a high performance team. Some people view ScrumMasters as process coaches, which is part of what they do, but they are really team coaches. For small teams, ScrumMasters can help on a part-time basis, which helps mitigate concerns about high budget overhead or staffing. And rather than trying to directly manage the work as a traditional project manager would, a ScrumMaster helps the team learn how to self-manage, providing guidance when necessary. What exactly would the ScrumMaster help the team with? This is where some of the practices used in Scrum can help deal with scaling factors.

The first Scrum practice that can help a team scale is the daily standup, which develops communication between developers (which is surprisingly hard if they are used to working alone) and helps focus their efforts. Speaking of tasks, the next Scrum practice that can help a team scale is to use a task board which helps visualize tasks queued in the backlog versus in progress versus done. This provides a focus for the daily standup discussions and helps provide a basic structure for the team to track their current progress. To forecast future progress the next Scrum practice that can help is the use of velocity and burndown charts to predict when milestone targets will be achieved. These practices help with scaling factors relating to communication and monitoring, but do not really help with quality issues. For that, the Scrum practice of definition of done is helpful, although the team will likely need coaching in how to adopt the way they work in order to achieve what they have defined as done.

The combination of these four practices is synergistic, imposes limited overhead, and is simple enough that developers can learn to do these practices on their own with some coaching from a ScrumMaster. Small teams of developers using these practices can therefore significantly mitigate the scaling factors that would otherwise cause issues.

One area I have struggled with is the role of ScrumMaster. The basic responsibilities are straight-forward, consisting of activities like facilitating meetings, dealing with impediments, and updating burndown charts. But, like Scrum, it is not a question of what to do as ScrumMaster but how to go about doing it. Over the past few years when I have had opportunities to interact with experienced ScrumMasters, I have noticed that they seem to have a different mindset when it comes to dealing with teams, especially in regards to the Agile principle of self-organizing teams.

So I was excited recently to have the opportunity to take a ScrumMaster course from Mark Levison. I made it my goal in the course to learn more about the mindset of great ScrumMasters and I wanted to share what I learned.

Growing a High Performance Team

One key insight I had came from a comment by Mark that Scrum is a method of growing a high performance team. On the surface, Scrum seems to be about producing a product, and thus it would make sense that the ScrumMaster's goal is to guide the team in creating a great product. But when you view Scrum as really being about building a great team, then the role of the ScrumMaster shifts correspondingly to that of helping the team grow and improve. This is, I believe, at the heart of what being a great ScrumMaster is.

The implication of this is that the ScrumMaster tries to minimize what they do directly and instead helps the team do things for themselves. Below are some examples of doing this that were discussed in the course:

• When dealing with impediments, the ScrumMaster should only directly work on them as a last resort. First, see if the team can handle the impediment on their own. If not, then see if the team can be coached through the resolution. Usually the only impediments that the ScrumMaster will take on themselves are external organizational problems far beyond the scope of the team. For example, if the team is having a problem with how a separate operations team is doing deployments, then rather than the ScrumMaster talking to ops on their own, bring along a team member, introduce them to the ops team, and enable them to work out their issues with ops directly.
• Prefer asking socratic questions of the team instead of telling the team what to do. For example, if the daily scrum is taking too long and is not focused enough, then rather than telling the team what to do raise this issue with the team, perhaps offering some suggestions, and let the team decide what to do.

By helping a team to do things on their own you build up their mastery, and by letting a team make their own decisions you build up their autonomy. Mastery and autonomy are two of the three drivers of internal motivation as per the book Drive: The Surprising Truth About What Motivates Us by Daniel Pink. If you always tell a team what to do and how to do it, they will not only fail to develop initiative but will develop a dependency upon you to provide all the answers, thus losing their innate initiative.

Servant Leader

I have heard the ScrumMaster role described as a servant leader. The ScrumMaster does not manage the team nor has formal authority over the team. Instead, they act as a humble steward supporting the team and product owner as depicted in the inverted organizational chart below.

Part of being a good leader is removing obstructions and obstacles from the way of the team and acting as a shield to protect the team from outside interference. This is part of being a ScrumMaster, but there is more to it than just this. For example, Mark described how one method he uses to assess a team is to sit in the team area and simply watch and listen as the team does their work. The following questions provide some examples of things to look for:

• How often are team members interacting with one another? Or do they stay silent in their cubicles?
• Who communicates with who? Do developers only talk to developers, or do they talk to the product owner (or business analyst)?
• What is the tone of the communication? How do testers communicate defects to developers and how do the developers respond? Are people positive or frustrated?
• How many interruptions are there from people external to the team?

This practice struck me as being identical to the Toyota principle of Genchi Genbutsu, translated as "go and see at the place where the work is done".

Continuous Improvement

Teams do not start off as high performing. Only through continual improvement can teams reach this state. Therefore a key responsibility of ScrumMasters is to cultivate a culture of continuous improvement and to encourage ongoing growth. Some tips for doing this:

• Put effort into the retrospective agenda to tailor it to the needs of the team and to adjust the activities from time to time to keep the energy level within the meeting high.
• Follow up on improvements identified within the retrospective to ensure they are actioned.
• When observation of the team indicates that changes are necessary, try suggesting the smallest change that will lead to improvement. Smaller changes cause less resistance and are easier to adopt.
• Encourage lots of experiments. Having the team commit to only trying a change for a limited period of time and then being able to evaluate afterwards whether to keep or discard the change overcomes a lot of resistance and can help cut through a lot of the debate over whether to adopt a change or not.
• Coach team members one-on-one regarding individual needs.

In conclusion, the ultimate objective of the ScrumMaster is to put themselves out of a job by elevating the team to such a high level of performance that they can take over all the ScrumMaster responsibilities.

I will start with statistics regarding the adoption of agile. The Forrester / Dr. Dobb's Q3 2010 Global Technographics Survey, as referenced in the Forrester paper Water-Scrum-Fall is the Reality of Agile for Most Organizations Today, found that 55% of projects where using some form of agile or iterative method. This statistic is now almost 2 years old. The Project Management Institute (PMI) reports that the use of Agile has tripled from 2008 to 2011 and states that Gartner predicts 80% of software development projects will be using Agile by the end of 2012.

Agile has often been perceived as developer-centric, so I have found it quite revealing that over the last year other I.T. roles have formally recognized the importance of Agile. The PMI has come out with its Agile Certified Practitioner certification and the International Institute of Business Analysis (IIBA) is finalizing their Agile Extension to the Business Analysis Body of Knowledge.

This information about the adoption of agile is interesting but I do not view it as sufficient evidence on its own to adopt agile. What I do find compelling is the benefits that Agile brings. This is clearly summarized by the 2011 CHAOS Manifesto from the Standish Group which states that "software applications developed through the agile process have three times the success rate of the traditional waterfall method and a much lower percentage of time and cost overruns".

I will conclude this article with one final reference that I believe perfectly captures the point of my article. Gartner's 2012 Planning Guide for Application Delivery Strategies has a section that is simply titled "Say goodbye to waterfall".

I'm giving a presentation on this topic Thursday, May 17th, 2012 at noon that aims to answer these questions and provide guidance based on a number of case studies involving Agile government projects. Along the way we will cover various perspectives on what Agile is, shatter some common misconceptions, and see some compelling evidence for why your next software development effort should use Agile.

The presentation is for Agile Edmonton at a new location - Startup Edmonton which is located on the 3rd floor of the historic Mercer Warehouse at 10363 – 104 Street (on the corner of 104st and 104 ave). Attendance is open to the public, so please drop by to attend the talk.

While the term agile itself is indeed ten years old, the philosophy and approach of iterative and incremental development (IID) has a surprisingly rich and extensive history. The article Iterative and Incremental Development: A Brief History (PDF) by Craig Larman and Victor R. Basili published in IEEE Computer discusses how the ideas of IID actually predate the existence of software, and have been used and promoted in every decade since for over 50 years. This article also discusses a classic 1970 article by Winston Royce well-known for supposedly promoting the use of waterfall, but which actually recommended the development of an initial pilot or preliminary version prior to creating the final version intended for delivery to the client. Larman's and Basili's article also discusses the ongoing evolution of the standard used by the U.S. Department of Defense for software development. The initial standard was document-heavy, gated, single-pass waterfall, but the high rate of project failures led to first the allowance of and eventually the full adoption of IID approaches.

So a limited set of organizations (often governments due to their byzantine contracting restrictions) did experience an evolution from waterfall to agile over time, but notably they started with a misunderstood version of waterfall. The use of IID approaches has always been part of the I.T. industry.

What about the effectiveness of agile? The second article Quality metrics: Software quality attributes and their rankings is an interview with Capers Jones and Olivier Bonsignour regarding their new book The Economics of Software Quality. The authors in this book discuss the effectiveness of over 100 quality factors using research based on over 10,000 software projects. On a scale of -10 for extremely harmful to a scale of +10 for extremely valuable, Agile methods rated a 9 - highly valuable - while waterfall only rated a 1 - barely useful.

These two articles highlight the fact that agile-like methods have been in use since the start of the software field, and are on average far more effective than the waterfall approach. This leads me to conclude that there is really no defensible reason for organizations to mandate or promote the use of waterfall.

There are a number of pragmatic practices that address this objective while avoiding the pitfalls afflicting formal traceability. I will start with a simpler form of traceability and then move on to practices seemingly unrelated to traceability.

Specification Highlighting to Track Test Coverage

A tester given a written specification to verify the software against must have some way of keeping track of their progress. One simple method is to highlight each statement in the specification as they test it. This is essentially tracking test coverage of the specification. I credit James Bach for this idea.

Testing Dashboard

A testing dashboard is great at illustrating overall testing progress and can be used as part of the summary in the final test report. To produce the dashboard first divided the system under test into test areas that usually correspond to features, components, or significant non-functional requirements. Then create a grid with the test areas as rows, and the columns report testing progress for each area. Key information to report per area is an assessment of quality (is this area ready to ship?), and the thoroughness of testing (test coverage and test effort). For an example and fuller explanation see James Bach's presentation on testing dashboards.

The testing dashboard is closest in form to a requirements traceability matrix - there's still a grid and something like requirements are listed down the grid. The big difference is that the dashboard shows a summary of testing for each area rather than listing individual tests and showing how they map.

Agile User Stories with Acceptance Tests

A common approach used by Agile methods is to divide requirements into fine-grained increments of business value called user stories and then define with examples or automated tests the criteria by which to accept that the story was correctly implemented. This takes a two-pronged approach to address traceability's objective. First, using fine-grained increments of functionality that are quickly developed minimizes the amount of work-in-progress that needs to be tracked and thus reduces the chance of mistakes. Second, the process produces tests for each user story early, sometimes before coding starts, so missing tests for a requirement is much less likely. Agile basically sufficiently changes how development is done to negate virtually all the sources of problems that would otherwise justify a requirements traceability matrix.

Task Tracking via Task Board

Popularized by Kanban but also used by many Agile teams, a task board is a practice for tracking the status of tasks. The board has multiple columns representing different status: the minimum set is usually "Not Started", "In Progress", and "Done". Tasks are placed on the board according to their status, and the team meets daily to discuss the tasks and update appropriately. I vastly prefer a physical task board on a wall, but for teams not co-located many online versions exist. Tasks are often quite fine-grained - one user story is often decomposed into multiple tasks.

Many teams include a status of "Testing" on the board, or use a symbol on the card to signify the completion of testing. Other quality control procedures such as code reviews can also be tracked. See the image below for an example of a task board with such states.

The combination of fine-grained tasks, daily updates, and tracking quality ensures that each requirement that is tackled by the team is properly developed. This achieves traceability's objective.

Conclusion

Despite formal traceability being too effort-intensive for too little value the underlying objective should not be ignored. I have described a variety of alternative practices that help ensure that requirements are properly developed and tested with a more modest level of effort.

Shakespeare quotes aside, every software development project has to make decisions on how many defects to fix and which ones to leave alone prior to shipping. While I have seldom seen this question debated within projects, the advice from industry thought leaders varies considerably. The Agile and Lean methods of software development in particular have somewhat opposing perspectives.

I believe that considering both sides of this question provides a fuller understanding of the issues and better equips us to answer appropriately. Therefore in the two sections below I explore the reasons behind both sides of the debate.

To Fix

• Shipping poor quality, defect-ridden code can upset users, turn away customers, and lead to a hard-to-shake bad reputation.
• The decision that a feature is worth developing is made with the expectation that it will work correctly. So any defects found in a feature means that the feature is still incomplete until these issues are fixed.
• Defects provide feedback regarding the development process. Each defect represents an opportunity to do a root cause analysis of what led to the defect and put countermeasures in place to prevent re-occurrence. The Lean mindset of "Stop the line" demands that new development be put on hold to fix newly discovered defects.
• Defects introduce the risk of compounding quality problems. The impact of a defect can be more significant than initially realized. Defects can be inadvertently replicated in other parts of the system. Enhancing components with too many defects can slow progress to a halt, as the system becomes essentially a shifting quicksand that is too unstable to work on. Constantly fixing defects helps maintain a high velocity of development over time.
• To mitigate risks in not fixing defects, each defect needs to be analyzed to understand its impact, cause, and required changes to fix. But after performing this analysis most of the work is usually done - the fix is relatively straightforward. Waiting to decide later to fix the defect (e.g. in a subsequent release) causes all the knowledge gained in the analysis to decay over time which is wasteful (in the Lean sense).

Not To Fix

• Significantly delaying the release of software to fix all defects leads to a loss of immediate revenue and potentially loss of market share due to competitors beating you to market. So you cannot afford to wait to fix all defects.
• The entrepreneurial mindset, especially for startups, is to ship early to get feedback from paying customers. Perfection is the enemy of the good.
• Under at least some versions of Scrum, defects are considered new tasks that are added to the product backlog to be prioritized by the product owner. This prioritization is based on the defect's impact (severity and likelihood of occurrence) and the effort required to fix it. Many minor defects will therefore likely never be fixed as new functionality will typically be of higher value.
• Stopping to analyze and fix defects disrupts developers who are in the middle of working on other functionality and is wasteful.
• Fixing defects in functionality that is already otherwise finished development and testing will require additional regression testing. Not fixing now and waiting until enhancements to this functionality are needed minimizes the extra effort required.

Conclusion

Shakespeare was wrong. There is actually a third perspective regarding whether or not to fix defects: avoid the question as much as possible by focusing on defect prevention. The Lean mindset of building quality in avoids all the waste associated with finding, analyzing, and fixing defects and should be our preferred approach. Only when it fails and the occasional defect is introduced do we then have to answer the question.

Adopting Test Driven Development

The boldest idea I heard was in response to my question of how to encourage the adoption of test driven development (TDD). The answer consisted of this process:

1. Clearly communicate to the team the expectation that all production code will be produced with tests.
2. Wait and monitor code check-ins until you see code checked in without tests.
3. Delete this untested code and commit the change.
4. Communicate to the affected developer that you found some code that was not necessary because the tests still passed after removing it. Diabolical laughter optional :)

This sounds extreme, but the team I was talking to actually used this in the formative early stages of building their team and the use of TDD has become a standard practice on the team. I think applying this at the start of a project with a new team is the best timing and makes the most sense, but I may have to try it mid-project (my team, watch out :)

I also learned of a different approach to writing individual tests that basically involves starting at the end and working backwards. Start first with the asserts that specify the desired result, then write the execution of the method under test which covers how the result will be obtained, and then finally write the setup code which provides the necessary inputs. This approach is very much a pull approach (in the lean sense) - each step serves as the specification for what needs to be done in the next step. As such, it might be an easier approach for newcomers to unit testing.

Adopting Pair Programming

I asked about dealing with resistance to adopting pair programming, especially the tendency of pairs to collaborate jointly for only a short time before reverting to working separately, each at their own computer. The answer was to take away someones computer, which leaves them with no choice but to pair with someone else on the team.

Environment factors can also help or hinder pairing. The Calgary teams have their workstations set up to allow people to sit side-by-side in front of the same machine. I think this layout is very important for successful long-duration pairing, and I switched my own cubicle over to this layout a while ago. The Calgary teams, however, took this setup one step further and have a second keyboard and mouse for each of their workstations. This has several benefits in terms of encouraging pairing:

• It communicates visually the expectation that two people will work at this computer rather than one.
• It eliminates concerns over using another person's keyboard. The 'owner' of that workstation has their dedicated keyboard, with the second keyboard reserved for guests. Whether the concern is over germs or over intruding on someones property/space, having only one keyboard produces a reluctance on the part of the visitor to drive.
• A common problem in pairing is for the person who is not driving to become too passive. Providing a second keyboard gives them the opportunity to jump in at any point (even when the driver does not want to relinquish control), and having the keyboard waiting in front of them is a subtle reminder to remain actively involved.

Using Task Boards

All the teams use task boards consisting of cards in various columns on a whiteboard, wall, or even a window. My team has recently switched to using a task board and I really like it in terms of it visualizing the work in progress and serving as a hub for discussions, like in the daily scrum. Given my experience and its broad use in Calgary, I am going to more actively promote its use.

Every team seems to have variations in how the task boards are used, but there were some commonalities across the Calgary teams. All teams used large cards - about the size of half a normal 8.5 x 11 page. Compared to the small post-it-note-sized cards my team uses this is a lot more real estate. These cards are made from colored paper cut in half that allows different colors to be used for different types of activities (e.g. defects, stories, and tasks). One team printed out defect reports (from JIRA) on regular paper and folded them in half to attach them to the wall. Another team attached printed requirements (with tests) folded in half to user stories. The teams all had relatively few columns for their task board - usually not more than four consisting of "In Sprint", "In Progress", "In Testing", and "Done".

Professional Development

The Calgary teams have a very strong culture of professional development that manifests in a number of ways. The most visible manifestation is my seeing many recent software development or I.T. books scattered around the office, many of which are books on my reading list (I may have to do an office raid :).

What I consider the most significant manifestation of this culture is the existence of a professional development group for senior developers. This group holds a number of activities that include periodic presentations by members on technical topics, book reading groups to jointly go through particular books, and Friday noon viewings of videos relating to software development.

One of the technical leads had an interesting rationale for doing professional development: not doing it is essentially stealing from your clients in the future. If you spend five percent of your time doing professional development and improve by five percent per year, then you will break even after the first year and show a benefit in future years: your output will be higher than before, even after subtracting the five percent. So not doing professional development will leave your client shortchanged in the future. If you let that trend continue long enough, you will eventually lose your client.

Summary

I had a great time and enjoyed being challenged with new ideas and perspectives on software development. I believe that visiting other teams or companies and seeing first-hand how they do things is quite beneficial. I encourage you to look for opportunities to do this and I hope to do more of it in the future. Thanks CGI Calgary!

Dealing with Uncertainty

Mike characterized traditional approaches as trying to manage out uncertainty in the effort to achieve predictable results. This strategy essentially involves determining up front the key aspects of the project (scope, budget, and schedule) and then seeking to minimize changes to it throughout the project. Unfortunately, virtually all software development is faced with a multitude of uncertainties. Trying to drive them out means conflicting with reality, which leads to a common litany of project problems such as: over budget, over schedule, scope achieved but business not satisfied (business objectives not met), and excessive cost for realized value.

Agile adopts the opposite philosophy: manage for uncertainty. Kent Beck used the term "embrace change" when first introducing Extreme Programming, and this is a common goal of all Agile methods. This is achieved through several strategies:

• Requirements are defined and implemented throughout the project, making it trivial to accommodate changes. The next section below discusses this in more detail.
• Providing frequent feedback via working software mitigates many of the risks of traditional development projects.
• Prioritizing user stories and first implementing the ones with the highest value / highest risk maximizes the return on investment and mitigates a number of common risks.
• Often Agile projects explicitly allow for the project to be stopped or brought to a close at any point if the business owner determines that there is no benefit in proceeding with the remaining requirements.

One of Mike's though-provoking points was that using an Agile method is a risk-mitigation strategy that mitigates many of the common risks associated with software development projects.

Deriving instead of Fixing Scope

One key aspect of project management is managing the iron triangle of scope, cost (budget), and time (schedule). In a traditional waterfall project the scope is defined first, the effort required to implement this scope is then estimated, and finally the budget and time required are derived from this estimate. This approach is maintained throughout the project: controls are often put in place to minimize changes to the scope and changes that are deemed necessary then typically require adjustments to the budget and/or schedule.

Fixing the scope up front is inherently flawed due to the nature of software development:

• Business requirements almost always change over time. I have seen statistics mentioned on the order of 1% change in requirements per month.
• Software development is inherently a learning activity. The business typically gains clarity about their requirements as they are analyzed and implemented. Insight is also gained into how the solution / system can best meet these requirements.
• One technique from lean software development is set-based concurrent engineering, which aims to defer decisions to the last possible moment when the most information is available. This helps ensure that the best possible decision is made and hence minimizes waste caused by bad decisions (one of the main objectives of lean). Fixing scope up front requires making decisions at the worst possible time – when the least is known.

Agile addresses this problem by making scope flexible. While an Agile project can (and usually does) start with a high-level backlog of features or epics, the specific user stories selected for implementation are determined throughout the project, iteration by iteration. I have heard some say that Agile makes the project scope easy to change, but in reality the scope is not being changed at all – it is being defined throughout the project. The total scope achieved in a project is hence a function of the project schedule: the more iterations and releases available, the more that can be done. Going back to the triangle of scope, cost, and time, this means that time is the main variable being controlled, from which budget and scope can be derived.

Conclusion

I really enjoyed the clarity of thinking Mike brought to the topic of Agile project management. Considering that Mike has given this presentation at major software development conferences that require $$$ to attend, I really appreciated having the opportunity to see it for free in my home town. I am a little surprised that there are not more people taking advantage of these opportunities. If you live in the Edmonton area I strongly encourage you to attend the Agile Edmonton meetings.

Whatever the work product is, using it for its intended purpose is the best way to discover if it is of sufficient quality to meet that purpose . For software, give it to actual users to use. For a requirements document, give it to developers to design the system. For a design document, give it to developers to write the code. For an operations manual, give it to support staff to maintain the system. If the user of the work product cannot easily get their tasks accomplished then the work product has failed the QA check and needs revision.

In my own experience I have lost track of the number of times that:

• Implementing a requirement has identified gaps, lack of clarity, or inconsistencies in carefully-reviewed and formally signed-off requirements documents.
• Coding up a documented design has identified gaping holes and deficiencies with the design that all the reviewers missed.
• Testing as per a carefully documented test plan reveals errors in the test scripts due to misunderstandings about how the system is supposed to work.
• Letting users finally use the software reveals misunderstandings or misinterpretations about the requirements that the entire testing process missed, or reveals that the user interface is confusing and non-intuitive.

The "Use It" method is not actually a new concept in the I.T. industry (the tone of the first paragraph is a little tongue-in-cheek). When developing software for developers there is the phrase "eating your own dog food" which refers to the practice of the development team using on a daily basis the software they are producing. Prototyping or proofs of concepts make use of a design in a limited fashion to verify some aspect of it. Beta release or limited production roll-outs provide software to actual users. Several principles of Agile development focus on getting software used in one way or another as per the following excerpts:

• ... early and continuous delivery of valuable software.
• Deliver working software frequently ...
• Working software is the primary measure of progress.

Successfully applying the "Use It" method of QA requires answering the following questions:

1. Who are the actual users of this work product?
2. How can we get these users to use the work product before it is finalized? If the actual users are completely unavailable then consider using proxies such as a business analyst or marketing representative.
3. How will we gather useful feedback on the quality of the work product from these users? This is a critical component to the method – without feedback no QA is being done.

As I stated at the beginning, reviews and testing are still necessary and important QA activities. But if you are not thinking about and planning how to have work products used and gather feedback from the users, you are missing a critical component in your QA strategy.