I use two guiding principles as the basis for constructing my definition.

1. Potentially releasable: Ideally the software can be released (or shipped) once it is done. I've seen many people, particularly in the context of Scrum, use the similar term "potentially shippable".
2. I would trust my life to my code: I just wrote an entire article on this titled Would You Trust Your Life To Your Code?

These principles are deliberately idealistic in order to set high expectations and motivate continuous improvement when I fall short of reaching them.

Different definitions of done can be created based on different levels or scopes. The two primary scopes are:

1. Done for a feature / user story.
2. Done for a release.

For this article I am using my definition of done for developing a feature (user story).

My definition of done is essentially a checklist with items grouped into categories. The lists of items and categories are not meant to dictate the process by which these items are done or the chronological order. For example, automated unit testing is listed in a separate category from coding but it is typically done at the same time or before-hand, if doing test-driven development.

Without further ado, here is my definition of done.

Coding

1. Code meets functional requirements.
2. Code meets non-functional requirements. Typical ones include:
   • Performance (capacity, scalability)
   • Usability
   • Security
   • Maintainability
3. Code is deployment-ready. This means environment-specific settings are extracted from the code base. A past article I wrote on designing for deployability provides more context on this.
4. Code is checked into version control.
5. Code complies with coding & architectural standards.
6. Code has been cleaned up. The goal is to ensure the code is easily readable and has a good design. In the past I have used the terms polishing code and refactoring to describe this. Robert C. Martin's book Clean Code: A Handbook of Agile Software Craftsmanship provides the best explanation of this that I have seen. Achieving this goes a long way towards meeting the maintainability requirement.
7. All TODO-style comments in the code have been addressed and removed.

Static Code Analysis

1. Code has been analyzed by static code analysis tools. The two primary tools I use for Java development are the Eclipse compiler and FindBugs. Other Java tools include PMD, Checkstyle, and Architecture Rules.
2. All errors and warnings found by the tools have either been corrected or have been suppressed with a comment indicating the reason for suppression.

Testing

1. Automated unit tests are written. The tests should be high quality (e.g. not brittle).
2. Automated integration tests are written that verify interactions with external systems such as the application database or third-party application / web services.
3. Code coverage achieved by the automated tests is measured and sufficient coverage is achieved. I use Cobertura for measuring code coverage. I do not like using only a numeric percentage coverage target as the sole definition of sufficient coverage, because this can encourage people to write poor-quality tests that merely execute the code rather than verify its correct behavior in order to meet the target. My true definition of sufficient coverage is that the tests execute and verify all code that could reasonably be incorrect. Having said this, I generally aim for at least 80% line (statement) coverage overall, and often achieve 90%+ coverage for individual classes. I am still debating what a reasonable target is for branch (conditional) coverage . I currently aim for at least 50% overall, but I have the feeling that a target of 75% would be better.
4. Functional testing by someone other than the developer has been done. Ideally this testing will be done by the customer, involve exercising the complete feature being coded in the way that users would use it, and be fully automated. More frequently I have seen this testing done manually (especially for user interfaces) by business analysts or testers who act as proxies for the customer. The key idea is to have someone other than the developer do testing to validate the assumptions and interpretations made (often implicitly) by the developer.

   I use the vague term "functional testing" rather than the more common terms "system testing" or "user acceptance testing" because projects can differ dramatically in what is done for system or user acceptance testing. If acceptance testing is done in a waterfall fashion as a separate phase near the end of the project then it cannot be part of the definition of done for a feature (but it is still part of the definition of done for the release). So I use the term "functional testing" to indicate this potential differentiation. Ideally, based on lean principles, all testing including system and user acceptance testing should be done as part of the work on a feature and not artificially delayed till later.

Reviewed

1. Design / approach has been reviewed by the technical lead / architect.
2. Detailed peer review / inspection has been done. If pair programming is being used then the peer review is automatically done at the same time as the coding. Otherwise, the reviewer should focus on issues that are less likely to be found by the static code analysis or automated testing. This can include items such as security holes, concurrency issues, and correctly meeting requirements.
3. Issues identified by reviewers have been resolved to the reviewer's satisfaction.

Other

1. Required documentation has been updated. This may include online help, user manual, or operations manual.
2. Build and deployment scripts and related configuration files have been updated.
3. No known defects are outstanding unless the customer has agreed to defer them, in which case they should be logged.
4. No known tasks related to the feature are outstanding.

That concludes my definition of done. I would appreciate hearing about what you use for a definition of done. In particular, if there is anything you think should be added or removed from my definition please let me know via a comment below.

The following graph is illustrative of the pattern:

Cause of Cyclical Incremental Growth

Lehman's general interpretation of these cycles was based on systems thinking in terms of feedback loops. Positive feedback – such as user requests for enhancements – leads to incremental growth in the next release. This in turn activates forces causing negative feedback – such as an increased number of defects to fix – which brings the growth back down. The following diagram illustrates these feedback loops causing a limit to growth:

If you are not familiar with the topic of systems thinking then I recommend the book The Fifth Discipline Fieldbook, which provides an insightful introduction to the topic of systems thinking and the use of these feedback diagrams.

Lehman discussed briefly possible causes of the cycles but did not explore this topic in depth. Based on my experience I believe that one primary cause of such cycles is that after significant enhancements to a system are released and put to use, defects, operational issues, and desirable minor changes are discovered. The next release(s) then focuses on addressing these issues, which leads to much lower incremental growth in the system. Once these issues are addressed, the subsequent release can focus more on adding features, which results in increased incremental growth and leads to the observed pattern of cyclic incremental growth.

I have observed this pattern occur multiple times with software I am involved in maintaining. One component at work I have been involved with for a few years now and consistently after every release with significant changes or additions to functionality there is a subsequent release to address defects, operational issues, or usability issues discovered through daily use of the component in production. I have seen a similar pattern with the open source software I distribute on my website. Often after releasing a new feature and using the software, I discover either a defect or a usability issue which I then fix and release a new version.

Preventing the Cycles

Can these cycles be prevented? In particular, can these issues found after the release goes to production be discovered instead through testing the system before release? I have no doubt that thorough testing would help reduce the number of issues found post-release. Extra testing will not, however, prevent the cycle. Software engineering research tells us that testing only finds a certain percentage of defects - usually under 50%. When I consider from my experience the types of issues discovered post-release, they fall into one of the following categories:

• Usability: The system functions as per the requirements and design, but is difficult or confusing for real users to use in practice.
• Maintenance issue: Production support staff experience difficulties supporting the operation of the application in production, i.e. due to insufficient logging or configuration options, or improper error handling.
• Incorrect requirements: Once users actually use the system, they discover that what they asked for (the requirements) is not exactly what they want.
• High-volume usage issues: I include in this category various types of issues that only seem to be discovered in production use including concurrency, compatibility, performance, and scalability.
• Functional Defect: A failure of the system to function as per the functional requirements and design.

While I would expect most functional defects to be caught prior to release with careful testing, most of the other categories are less likely. To even have a reasonable chance of discovering these other issues requires a special focus on these categories during development and testing – i.e. including usability testing or stress testing. Given that functional testing is frequently cut short to meet project deadlines, these other types of testing are much less likely to be done at all, let alone thoroughly.

The fundamental issue is that the most thorough way to test software is to put it into active, daily use. (This is not necessarily the most cost effective way nor is it best for the users.) So despite all the testing that may be done prior to release, deploying the software to production and letting users hammer it is going to reveal issues. I believe that this leads, inevitably, to the observed cycles of incremental growth, as one or more follow-up releases are needed to address defects. The amplitude and frequency of the cycles can be reduced, but not eliminated.

Lessons Learned

What lessons can be learned from this pattern of cyclical incremental growth? What common practices in I.T. try to address this cycle?

• For risk-adverse organizations, avoid software that has just been released with significant enhancements – i.e. release X.0. Instead wait for subsequent minor releases that address the inevitable issues found after the release – i.e. release X.0.1 or X.1.
• Use beta releases to simulate an actual release. Whether the beta testers are a handpicked subset of existing users or are early adopters who voluntarily download the beta, the idea is to put the software into hands of users with the agreed expectation that there will be issues. This needs to balance two competing forces: putting the software to use as widely as possible to find issues versus keeping users happy about the product – which finding defects makes more difficult.
• Software development teams should plan for an issue-fixing release after major enhancement releases, especially if other steps are not taken to minimize the cyclical pattern. This is surprisingly difficult as it is far easier for the team and management to stay in denial ("the software has no defects" being a common refrain of developers) than to accept reality and plan accordingly.

Why do I feel that the release when ready strategy is generally best? The objective behind creating software is to have it fulfill a function, and to do this it must work properly. My definition of ready software is essentially a more precise way of saying that the software works. If the software does not work properly, why would you release it prematurely? There are a variety of factors responsible for the large amounts of premature software being released:

• Politics: A mandated budget or release date with no basis in reality or an arbitrary reduction in the schedule after it has been estimated are both problematic. In fact, such practices could be considered a fourth release 'strategy': release by management fiat. A more subtle case when politics has a negative impact is when it is onerous to change the budget or release date due to the bureaucracy and paperwork involved.
• Fixed Resources: There are situations where the amount of time or budget is fixed and cannot be changed. One example is software changes due to new legislation that comes into effect at a certain date, such as the recent change in the start time of daylight saving time. Startups often have limited funds, and need to have a software release to market before that money is exhausted.
• Competition: There is a commonly-held perception that being first-to-market is a competitive advantage, and this belief causes marketing departments to push for accelerated release schedules.
• Demand for Features: For both commercial and enterprise software, consumers and users generally demand more features over quality. After people start using the software and discovering problems, then they might begin to demand higher quality. Even then, sometimes the users are not the purchasers, as is the case for most business and enterprises software. This demand for more features influences development teams to include just one more feature rather than spend the time necessary to ensure the software is ready for release (i.e. via code reviews and testing).
• Ignorance: Managers who do not understand software development may come to believe that the software is ready to release when it really is not. This is a common risk of producing software prototypes.

These factors can make it difficult to impossible for a development team to apply the release when ready strategy. It is possible, however, to adapt to a release by schedule (or budget) strategy while maximizing the odds of the software being ready. To do this, you first must understand project schedules - in particular how the interrelated factors of time, resources, scope, and quality affect the schedule. Releasing software when ready implies that the quality is high, and a release by schedule strategy implies that the time is fixed. Adding resources is one option, but this is ineffective in the short term (due to Brook's law), difficult to accomplish quickly, and it does not help when the budget is fixed. This leaves scope as the single factor remaining. The key then is to carefully manage the scope throughout the entire development effort in a variety of ways:

• Prioritize requirements and features into must-haves, important, and nice-to-haves. Work on the highest priority features first.
• Structure and organize features into self-contained units to make it possible to drop features that cannot be finished by the release date.
• Fully complete features before moving on to new features, especially as the release date draws near. This means that the work activities should not be structured by technologies or application layers, but by end-to-end functionality. For example, rather than working on the database persistence layer first for all the features, and then turning to the business logic and user interface, do all three layers for a single feature only.
• Track the progress on each feature according to whether it is ready for release or not. This means that a feature is not done when the developer is done coding, but is only after it has been reviewed, tested, and all the serious defects eliminated. This implies that you do not wait till the end of the project to do all the code reviews and testing. Tracking progress by ready-to-release features allows you to still release working software if you run out of time by dropping all the incomplete features.
• Define a project milestone sufficiently in advance of the release date for making the final decision as to what features will make the release, and which features will be dropped. This allows the time required to remove dropped features from the code base or disable them and to perform 'final' testing on the release candidate.

While the above points can help deal with a fixed release date, I feel it is far from the ideal situation. I believe the best approach is the release when ready strategy. You can still have a target release date, but make it flexible. You still aim for that release date using the points above for managing scope, but making the date flexible allows you to take a more rational and balanced approach. As the release date approaches, you can decide whether to cut features or push out the release date. During final testing, if too many defects are being found, you can decide to spend more time improving the quality of the software and push out the release date.

My most significant growth was in the area of personal productivity: I read and implemented the organizational system described in the book Getting Things Done : The Art Of Stress-Free Productivity by David Allen. I have found this system very useful both at work and at home, and wrote an article describing my experience implementing the system.

This last year I switched to a four day work week, and have found it to be a significant improvement over the normal five days. I even figured out how to do this without a drop in pay.

One area of software development I have explored a lot this year has been change and release management: specifically the process of promoting application or infrastructure changes into a production environment. Both my experiences at work and my experiences running this website have made me appreciate the necessity of having a defined process. The amount of process that is necessary depends on the types of changes being made and the complexity of the environment. As I wrote in my article on deploying application changes, it is easier when the application is packaged and deployed as a single unit. When this is not possible - like for database changes - then more process is necessary. Adding too much process, however, can be just as harmful as having too little - a proper balance must be maintained.

Over this last year I have spent a lot of time thinking about application reliability. Ensuring a system is highly reliable is a surprisingly difficult task, and one of the major culprits is complexity. I have found myself arguing more strongly for simpler, more reliable solutions as a result. My most recent focus has been on error handling. My article on the fail fast and degrade gracefully approaches to error handling contains some earlier thoughts on this subject, but I have not yet written up my latest ideas. I have come to believe that the degrade gracefully approach, while more difficult to implement, is the best for creating highly reliable software. I have unfortunately encountered a third approach to error handling: ignore errors. This leads to silent, undetected failures whose negative effects go unnoticed for an arbitrary length of time.

I have also learned a number of lessons from running this website, including improving my knowledge of web design, promotion, and on-line advertising. I have found the process of writing my weekly articles to be a great learning aid. Writing forces me to reflect on and clarify my thoughts and ideas on a particular subject. It is no coincidence that I have provided a number of links above to articles I have written this past year. I tend to write about topics that are currently in my focus of attention, and the act of writing about them helps clarify and solidify the learning I have done.

This application deployment process must specify the set of application artifacts (i.e. executable code, database objects, files) that is deployed for a particular release - what I call the unit of deployment. There are two basic approaches to defining a unit of deployment.

The first approach is a full deployment: packaging together the entire application as a single unit. As a Java developer, this seems like a natural approach: the compiled Java class files along with other application files can be bundled into a single JAR, WAR or EAR file. A typical enterprise business application, however, consists of more than just the code. For the database in particular, deploying all the tables and views for the entire application when doing a release is not practical. Therefore, the second approach to defining a unit of deployment is a partial deployment: only include those application artifacts that have changed. The remainder of the artifacts are left unmodified in the environment being deployed to. For the database, this means that only those tables or views that have been modified need to be included in the deployment.

These two approaches each have advantages and disadvantages. The following criteria are useful for determining which approach should be used in a particular situation:

• How confidently can you track the application changes? If it is hard or error-prone to track changes to artifacts, then a full deployment is safer. With a partial deployment, you need to know that all relevant changes made in the development environment are being promoted to the test environment, and then from the test to the production environment. Missing one or more artifacts in the deployment to production can cause defects or even a complete failure of the application.
• How much manual effort is required to do the deployment? Manual effort increases the likelihood of a failed deployment due to human error. A full deployment is easy to automate and should require little if any manual effort to maintain, since you always just grab the entire application. A partial deployment can be automated for a single change, but it is hard to have a reusable automatic deployment mechanism since the set of artifacts that change will vary from release to release. Such a mechanism needs to be able to determine automatically what has changed and deploy just those pieces. A good example of this is the automatic update mechanisms found in operating systems and applications - they only download a patch containing the changes rather than an entire copy of the updated application.
• How much time is required to do the deployment? The longer it takes to deploy a single artifact, the more important this criteria, and the more valuable the partial deployment approach becomes. Database changes are the best example: to drop and recreate every table in the database for a change would require a full backup and restore of the data, which will often take prohibitively longer than simply updating the few tables that actually changed. Online application updates are limited by network bandwidth, so they also benefit from a partial deployment.

My preference is for full deployments. Unless circumstances require a partial deployment, I feel that the default choice should be to deploy everything. Unfortunately, this is seldom possible in a complex enterprise environment. Many enterprise applications consist of a collection of components or services such as web application, database, batch processing, scheduling, reporting, and web services. Due to organizational or logistical reasons, it is seldom possible to do a full deployment of all of these components in a single release. At the application level, therefore, a partial deployment is often done. It is at the component level where it is easier to achieve full deployments of the individual components.