While the term agile itself is indeed ten years old, the philosophy and approach of iterative and incremental development (IID) has a surprisingly rich and extensive history. The article Iterative and Incremental Development: A Brief History (PDF) by Craig Larman and Victor R. Basili published in IEEE Computer discusses how the ideas of IID actually predate the existence of software, and have been used and promoted in every decade since for over 50 years. This article also discusses a classic 1970 article by Winston Royce well-known for supposedly promoting the use of waterfall, but which actually recommended the development of an initial pilot or preliminary version prior to creating the final version intended for delivery to the client. Larman's and Basili's article also discusses the ongoing evolution of the standard used by the U.S. Department of Defense for software development. The initial standard was document-heavy, gated, single-pass waterfall, but the high rate of project failures led to first the allowance of and eventually the full adoption of IID approaches.

So a limited set of organizations (often governments due to their byzantine contracting restrictions) did experience an evolution from waterfall to agile over time, but notably they started with a misunderstood version of waterfall. The use of IID approaches has always been part of the I.T. industry.

What about the effectiveness of agile? The second article Quality metrics: Software quality attributes and their rankings is an interview with Capers Jones and Olivier Bonsignour regarding their new book The Economics of Software Quality. The authors in this book discuss the effectiveness of over 100 quality factors using research based on over 10,000 software projects. On a scale of -10 for extremely harmful to a scale of +10 for extremely valuable, Agile methods rated a 9 - highly valuable - while waterfall only rated a 1 - barely useful.

These two articles highlight the fact that agile-like methods have been in use since the start of the software field, and are on average far more effective than the waterfall approach. This leads me to conclude that there is really no defensible reason for organizations to mandate or promote the use of waterfall.

There are a number of pragmatic practices that address this objective while avoiding the pitfalls afflicting formal traceability. I will start with a simpler form of traceability and then move on to practices seemingly unrelated to traceability.

Specification Highlighting to Track Test Coverage

A tester given a written specification to verify the software against must have some way of keeping track of their progress. One simple method is to highlight each statement in the specification as they test it. This is essentially tracking test coverage of the specification. I credit James Bach for this idea.

Testing Dashboard

A testing dashboard is great at illustrating overall testing progress and can be used as part of the summary in the final test report. To produce the dashboard first divided the system under test into test areas that usually correspond to features, components, or significant non-functional requirements. Then create a grid with the test areas as rows, and the columns report testing progress for each area. Key information to report per area is an assessment of quality (is this area ready to ship?), and the thoroughness of testing (test coverage and test effort). For an example and fuller explanation see James Bach's presentation on testing dashboards.

The testing dashboard is closest in form to a requirements traceability matrix - there's still a grid and something like requirements are listed down the grid. The big difference is that the dashboard shows a summary of testing for each area rather than listing individual tests and showing how they map.

Agile User Stories with Acceptance Tests

A common approach used by Agile methods is to divide requirements into fine-grained increments of business value called user stories and then define with examples or automated tests the criteria by which to accept that the story was correctly implemented. This takes a two-pronged approach to address traceability's objective. First, using fine-grained increments of functionality that are quickly developed minimizes the amount of work-in-progress that needs to be tracked and thus reduces the chance of mistakes. Second, the process produces tests for each user story early, sometimes before coding starts, so missing tests for a requirement is much less likely. Agile basically sufficiently changes how development is done to negate virtually all the sources of problems that would otherwise justify a requirements traceability matrix.

Task Tracking via Task Board

Popularized by Kanban but also used by many Agile teams, a task board is a practice for tracking the status of tasks. The board has multiple columns representing different status: the minimum set is usually "Not Started", "In Progress", and "Done". Tasks are placed on the board according to their status, and the team meets daily to discuss the tasks and update appropriately. I vastly prefer a physical task board on a wall, but for teams not co-located many online versions exist. Tasks are often quite fine-grained - one user story is often decomposed into multiple tasks.

Many teams include a status of "Testing" on the board, or use a symbol on the card to signify the completion of testing. Other quality control procedures such as code reviews can also be tracked. See the image below for an example of a task board with such states.

The combination of fine-grained tasks, daily updates, and tracking quality ensures that each requirement that is tackled by the team is properly developed. This achieves traceability's objective.

Conclusion

Despite formal traceability being too effort-intensive for too little value the underlying objective should not be ignored. I have described a variety of alternative practices that help ensure that requirements are properly developed and tested with a more modest level of effort.

Shakespeare quotes aside, every software development project has to make decisions on how many defects to fix and which ones to leave alone prior to shipping. While I have seldom seen this question debated within projects, the advice from industry thought leaders varies considerably. The Agile and Lean methods of software development in particular have somewhat opposing perspectives.

I believe that considering both sides of this question provides a fuller understanding of the issues and better equips us to answer appropriately. Therefore in the two sections below I explore the reasons behind both sides of the debate.

To Fix

• Shipping poor quality, defect-ridden code can upset users, turn away customers, and lead to a hard-to-shake bad reputation.
• The decision that a feature is worth developing is made with the expectation that it will work correctly. So any defects found in a feature means that the feature is still incomplete until these issues are fixed.
• Defects provide feedback regarding the development process. Each defect represents an opportunity to do a root cause analysis of what led to the defect and put countermeasures in place to prevent re-occurrence. The Lean mindset of "Stop the line" demands that new development be put on hold to fix newly discovered defects.
• Defects introduce the risk of compounding quality problems. The impact of a defect can be more significant than initially realized. Defects can be inadvertently replicated in other parts of the system. Enhancing components with too many defects can slow progress to a halt, as the system becomes essentially a shifting quicksand that is too unstable to work on. Constantly fixing defects helps maintain a high velocity of development over time.
• To mitigate risks in not fixing defects, each defect needs to be analyzed to understand its impact, cause, and required changes to fix. But after performing this analysis most of the work is usually done - the fix is relatively straightforward. Waiting to decide later to fix the defect (e.g. in a subsequent release) causes all the knowledge gained in the analysis to decay over time which is wasteful (in the Lean sense).

Not To Fix

• Significantly delaying the release of software to fix all defects leads to a loss of immediate revenue and potentially loss of market share due to competitors beating you to market. So you cannot afford to wait to fix all defects.
• The entrepreneurial mindset, especially for startups, is to ship early to get feedback from paying customers. Perfection is the enemy of the good.
• Under at least some versions of Scrum, defects are considered new tasks that are added to the product backlog to be prioritized by the product owner. This prioritization is based on the defect's impact (severity and likelihood of occurrence) and the effort required to fix it. Many minor defects will therefore likely never be fixed as new functionality will typically be of higher value.
• Stopping to analyze and fix defects disrupts developers who are in the middle of working on other functionality and is wasteful.
• Fixing defects in functionality that is already otherwise finished development and testing will require additional regression testing. Not fixing now and waiting until enhancements to this functionality are needed minimizes the extra effort required.

Conclusion

Shakespeare was wrong. There is actually a third perspective regarding whether or not to fix defects: avoid the question as much as possible by focusing on defect prevention. The Lean mindset of building quality in avoids all the waste associated with finding, analyzing, and fixing defects and should be our preferred approach. Only when it fails and the occasional defect is introduced do we then have to answer the question.

Adopting Test Driven Development

The boldest idea I heard was in response to my question of how to encourage the adoption of test driven development (TDD). The answer consisted of this process:

1. Clearly communicate to the team the expectation that all production code will be produced with tests.
2. Wait and monitor code check-ins until you see code checked in without tests.
3. Delete this untested code and commit the change.
4. Communicate to the affected developer that you found some code that was not necessary because the tests still passed after removing it. Diabolical laughter optional :)

This sounds extreme, but the team I was talking to actually used this in the formative early stages of building their team and the use of TDD has become a standard practice on the team. I think applying this at the start of a project with a new team is the best timing and makes the most sense, but I may have to try it mid-project (my team, watch out :)

I also learned of a different approach to writing individual tests that basically involves starting at the end and working backwards. Start first with the asserts that specify the desired result, then write the execution of the method under test which covers how the result will be obtained, and then finally write the setup code which provides the necessary inputs. This approach is very much a pull approach (in the lean sense) - each step serves as the specification for what needs to be done in the next step. As such, it might be an easier approach for newcomers to unit testing.

Adopting Pair Programming

I asked about dealing with resistance to adopting pair programming, especially the tendency of pairs to collaborate jointly for only a short time before reverting to working separately, each at their own computer. The answer was to take away someones computer, which leaves them with no choice but to pair with someone else on the team.

Environment factors can also help or hinder pairing. The Calgary teams have their workstations set up to allow people to sit side-by-side in front of the same machine. I think this layout is very important for successful long-duration pairing, and I switched my own cubicle over to this layout a while ago. The Calgary teams, however, took this setup one step further and have a second keyboard and mouse for each of their workstations. This has several benefits in terms of encouraging pairing:

• It communicates visually the expectation that two people will work at this computer rather than one.
• It eliminates concerns over using another person's keyboard. The 'owner' of that workstation has their dedicated keyboard, with the second keyboard reserved for guests. Whether the concern is over germs or over intruding on someones property/space, having only one keyboard produces a reluctance on the part of the visitor to drive.
• A common problem in pairing is for the person who is not driving to become too passive. Providing a second keyboard gives them the opportunity to jump in at any point (even when the driver does not want to relinquish control), and having the keyboard waiting in front of them is a subtle reminder to remain actively involved.

Using Task Boards

All the teams use task boards consisting of cards in various columns on a whiteboard, wall, or even a window. My team has recently switched to using a task board and I really like it in terms of it visualizing the work in progress and serving as a hub for discussions, like in the daily scrum. Given my experience and its broad use in Calgary, I am going to more actively promote its use.

Every team seems to have variations in how the task boards are used, but there were some commonalities across the Calgary teams. All teams used large cards - about the size of half a normal 8.5 x 11 page. Compared to the small post-it-note-sized cards my team uses this is a lot more real estate. These cards are made from colored paper cut in half that allows different colors to be used for different types of activities (e.g. defects, stories, and tasks). One team printed out defect reports (from JIRA) on regular paper and folded them in half to attach them to the wall. Another team attached printed requirements (with tests) folded in half to user stories. The teams all had relatively few columns for their task board - usually not more than four consisting of "In Sprint", "In Progress", "In Testing", and "Done".

Professional Development

The Calgary teams have a very strong culture of professional development that manifests in a number of ways. The most visible manifestation is my seeing many recent software development or I.T. books scattered around the office, many of which are books on my reading list (I may have to do an office raid :).

What I consider the most significant manifestation of this culture is the existence of a professional development group for senior developers. This group holds a number of activities that include periodic presentations by members on technical topics, book reading groups to jointly go through particular books, and Friday noon viewings of videos relating to software development.

One of the technical leads had an interesting rationale for doing professional development: not doing it is essentially stealing from your clients in the future. If you spend five percent of your time doing professional development and improve by five percent per year, then you will break even after the first year and show a benefit in future years: your output will be higher than before, even after subtracting the five percent. So not doing professional development will leave your client shortchanged in the future. If you let that trend continue long enough, you will eventually lose your client.

Summary

I had a great time and enjoyed being challenged with new ideas and perspectives on software development. I believe that visiting other teams or companies and seeing first-hand how they do things is quite beneficial. I encourage you to look for opportunities to do this and I hope to do more of it in the future. Thanks CGI Calgary!

Dealing with Uncertainty

Mike characterized traditional approaches as trying to manage out uncertainty in the effort to achieve predictable results. This strategy essentially involves determining up front the key aspects of the project (scope, budget, and schedule) and then seeking to minimize changes to it throughout the project. Unfortunately, virtually all software development is faced with a multitude of uncertainties. Trying to drive them out means conflicting with reality, which leads to a common litany of project problems such as: over budget, over schedule, scope achieved but business not satisfied (business objectives not met), and excessive cost for realized value.

Agile adopts the opposite philosophy: manage for uncertainty. Kent Beck used the term "embrace change" when first introducing Extreme Programming, and this is a common goal of all Agile methods. This is achieved through several strategies:

• Requirements are defined and implemented throughout the project, making it trivial to accommodate changes. The next section below discusses this in more detail.
• Providing frequent feedback via working software mitigates many of the risks of traditional development projects.
• Prioritizing user stories and first implementing the ones with the highest value / highest risk maximizes the return on investment and mitigates a number of common risks.
• Often Agile projects explicitly allow for the project to be stopped or brought to a close at any point if the business owner determines that there is no benefit in proceeding with the remaining requirements.

One of Mike's though-provoking points was that using an Agile method is a risk-mitigation strategy that mitigates many of the common risks associated with software development projects.

Deriving instead of Fixing Scope

One key aspect of project management is managing the iron triangle of scope, cost (budget), and time (schedule). In a traditional waterfall project the scope is defined first, the effort required to implement this scope is then estimated, and finally the budget and time required are derived from this estimate. This approach is maintained throughout the project: controls are often put in place to minimize changes to the scope and changes that are deemed necessary then typically require adjustments to the budget and/or schedule.

Fixing the scope up front is inherently flawed due to the nature of software development:

• Business requirements almost always change over time. I have seen statistics mentioned on the order of 1% change in requirements per month.
• Software development is inherently a learning activity. The business typically gains clarity about their requirements as they are analyzed and implemented. Insight is also gained into how the solution / system can best meet these requirements.
• One technique from lean software development is set-based concurrent engineering, which aims to defer decisions to the last possible moment when the most information is available. This helps ensure that the best possible decision is made and hence minimizes waste caused by bad decisions (one of the main objectives of lean). Fixing scope up front requires making decisions at the worst possible time – when the least is known.

Agile addresses this problem by making scope flexible. While an Agile project can (and usually does) start with a high-level backlog of features or epics, the specific user stories selected for implementation are determined throughout the project, iteration by iteration. I have heard some say that Agile makes the project scope easy to change, but in reality the scope is not being changed at all – it is being defined throughout the project. The total scope achieved in a project is hence a function of the project schedule: the more iterations and releases available, the more that can be done. Going back to the triangle of scope, cost, and time, this means that time is the main variable being controlled, from which budget and scope can be derived.

Conclusion

I really enjoyed the clarity of thinking Mike brought to the topic of Agile project management. Considering that Mike has given this presentation at major software development conferences that require $$$ to attend, I really appreciated having the opportunity to see it for free in my home town. I am a little surprised that there are not more people taking advantage of these opportunities. If you live in the Edmonton area I strongly encourage you to attend the Agile Edmonton meetings.

Whatever the work product is, using it for its intended purpose is the best way to discover if it is of sufficient quality to meet that purpose . For software, give it to actual users to use. For a requirements document, give it to developers to design the system. For a design document, give it to developers to write the code. For an operations manual, give it to support staff to maintain the system. If the user of the work product cannot easily get their tasks accomplished then the work product has failed the QA check and needs revision.

In my own experience I have lost track of the number of times that:

• Implementing a requirement has identified gaps, lack of clarity, or inconsistencies in carefully-reviewed and formally signed-off requirements documents.
• Coding up a documented design has identified gaping holes and deficiencies with the design that all the reviewers missed.
• Testing as per a carefully documented test plan reveals errors in the test scripts due to misunderstandings about how the system is supposed to work.
• Letting users finally use the software reveals misunderstandings or misinterpretations about the requirements that the entire testing process missed, or reveals that the user interface is confusing and non-intuitive.

The "Use It" method is not actually a new concept in the I.T. industry (the tone of the first paragraph is a little tongue-in-cheek). When developing software for developers there is the phrase "eating your own dog food" which refers to the practice of the development team using on a daily basis the software they are producing. Prototyping or proofs of concepts make use of a design in a limited fashion to verify some aspect of it. Beta release or limited production roll-outs provide software to actual users. Several principles of Agile development focus on getting software used in one way or another as per the following excerpts:

• ... early and continuous delivery of valuable software.
• Deliver working software frequently ...
• Working software is the primary measure of progress.

Successfully applying the "Use It" method of QA requires answering the following questions:

1. Who are the actual users of this work product?
2. How can we get these users to use the work product before it is finalized? If the actual users are completely unavailable then consider using proxies such as a business analyst or marketing representative.
3. How will we gather useful feedback on the quality of the work product from these users? This is a critical component to the method – without feedback no QA is being done.

As I stated at the beginning, reviews and testing are still necessary and important QA activities. But if you are not thinking about and planning how to have work products used and gather feedback from the users, you are missing a critical component in your QA strategy.

But then I read an interview with Alastair Cockburn in which he talked about what it means to be agile. Cockburn provided a top ten list of indicators that you are not agile. As I read this list, I realized with a sinking feeling that most of these applied to my current team. I went back through the list and recorded my team's score out of 10 for the indicators for which we were agile. I even gave us half a point for those items for which we were partially agile. Our final score was a measly 3 out of 10 (10 being fully agile). Clearly we are not agile.

Would I have to revoke my agile membership? I still believed in the agile approach, so how could I not be agile? Or was I mistaken? I re-read the Agile Manifesto, and still agreed with what it said. What about the principles of agile software? I re-read those as well, and again found myself in agreement. There were a few principles that are difficult or impossible to apply in my current environment, but that has not stopped me from trying to do what I can.

I agree with the agile values and principles and try to apply them within my work environment. I believe this is the fundamental indicator of whether you as an individual are agile or not, rather than a list of specific practices. Therefore I am agile.

A separate question is whether I, as part of a team, am doing agile software development. Despite the flaws with Cockburn's top ten list, it does help evaluate this second question. The answer for myself is no. I am an agile software developer doing non-agile software development. The distinction may be subtle, but it is significant. To help you see it, imagine the opposite case of a non-agile developer working on an agile software development process. They may be following the team's agile processes while pushing for more formal documentation and more detailed processes. Your team's approach does not necessarily reflect the beliefs and values of each team member. Another way of viewing this distinction is that agile can be thought of as both a philosophy consisting of a set of values and principles, and as a set of practices which implement and uphold these principles and values. I am unable to follow all of the practices at work, but I still subscribe to and apply the values and principles.