Defining what makes for a successful project / product is critically important at every level: the individual, the team, the organization, and even the entire I.T. industry. (In the remainder of this article I will refer solely to projects with the understanding that the material applies as well to ongoing work on software products.)

Individual

Individual performance is often evaluated at least in part by the success or failure of the projects you work on, more so if you are in a leadership or management role. This can impact your compensation (raises, bonuses), future assignments and promotions, and even your job (if the project is a bad failure).

Being clear about the project’s definition of success should help guide you in the day-to-day decisions you make and actions you take on the project. This provides two benefits: you are more likely to be personally judged a success (even if the project fails), and the project is more likely to be successful.

Team

The choices to make in assembling a team – how many people will be on it, and what skill levels & abilities will they have – are significantly influenced by the project’s definition of success. For example, if minimal budget expenditure is a critical success factor then a small team should be used.

Once the team is assembled having a clear definition of success shared by the team members helps provide coordination of activity rather than disjointed efforts.

The practices & methods used by the team should be selected and tailored to maximize the likelihood and magnitude of success.

Organization

Within an organization that produces software decisions must be made regarding what projects to proceed or continue with and what level of funding and people to allocate to these projects. In a rational organization these decisions should be based on a cost-benefit analysis which should, in turn, be linked to the project’s definition of success. For example, if a project is conceived of to produce software targeted to hit the market in time for the Christmas selling season then hitting the delivery date becomes a significant factor of project success.

I.T. Industry

Our favorite pastime within the I.T. industry may well be promoting or debating the merits of various programming languages, environments, tools, practices, and methods. But what is the basis for making these evaluations? Presumably this is based on the likelihood of the tool, practice, or method making the project more successful (or less likely to fail). This evaluation should depend, therefore, on the definition of success being used, but this is seldom defined explicitly as part of the promotion / evaluation being done. (I am as guilty of this as everyone else.)

One notable exception is the book Rapid Development by Steve McConnell. Part III of the book lists 27 ‘best’ practices. For each practice a section on efficacy is provided that evaluates the practice based on three separate factors: potential reduction from nominal schedule, improvement in progress visibility, and effect on schedule risk. (Other project success factors such as quality or cost were omitted because the book focuses on minimizing schedule – thus the name Rapid Development.) McConnell explicitly advocates “selecting rapid-development practices that meet the needs of your specific project” (page 396).

Some might argue that virtually all projects share the same definition of success (usually along the lines of delivering required functionality with sufficient quality on time and on budget), and this then allows for meaningful discussion of 'best' practices independent of the definition of project success. I disagree with this position, but will save further discussion for a subsequent article: Problems with Typical Definitions of Project Success

Benefits of Rotations

The use of rotations provides two key benefits:

1. Rapid learning: People learn most rapidly when placed into new, unfamiliar situations, as opposed to being in the same environment year after year. The initial ramp-up period for new team members often corresponds to the period of increased learning, after which it tapers down. The use of rotations essentially maintains this ramp-up state to realize this enhanced learning on a more ongoing basis.
2. Breadth of experience: Rotation assignments are specifically selected to provide a broad range of experience in the field. This helps trainees determine what they would like to specialize in, and once specialized helps them work more effectively with other specializations. Breadth of experience helps develop that "big picture" understanding of the field and provides better appreciation for how each specialization contributes to it.

While software development is quite a different field from medicine, I believe that the use of rotations is just as beneficial to developing expertise as developers. One example is spending time maintaining or enhancing a preexisting code base tends to teach the importance of having cleanly design and easily understood code – in other words maintainability. Another more general example is developing that breadth of experience tends to provide a better appreciation of the importance of context in determining what development technologies, practices, and methodologies to apply.

Required Rotations

What kinds of rotations would be beneficial for developers? Listed below are the key assignments that I feel every developer should experience:

• Production Support: Keep an operational system running. This develops your ability to diagnose and fix problems. It teaches you about all the bizarre events that can happen in a real system that never seem to happen in testing. I personally found it humbling to watch systems I thought were solid get hammered and fail in ways I never expected. You will learn the value of many non-functional requirements such as reliability, performance, scalability, manageability, and supportability (e.g. good logging).
• Maintenance: Make enhancements and fixes to an existing code base in active use. You should do this both for a system you originally developed and one you have not. This teaches the importance of good design and readable code – i.e. maintainability.
• End-User Support: Directly dealing with user inquiries and problems. This is enlightening in numerous ways: it provides you with a better understanding of what users value, how they react, their level of understanding of I.T., and what they like and do not like in the software they are using.
• New Development: Develop a piece of software going through all the stages of the software development life cycle: inception, requirements, design, coding, testing, release, and maintenance. Going through the entire cycle gives you a deeper understanding of the big picture. More importantly, you will have the opportunity to benefit from feedback in later stages regarding earlier stages: e.g. coding indicates requirements were poorly understood by all, or testing reveals a design defect.

Optional Rotations

There are many other possible rotations across the full breadth of the field of software development. A large (but not complete) list of these assignments is provided below, organized into categories.

• Assignments in different types of software development organizations:
  • Product-focused
  • Service-focused (e.g. consulting)
  • Internal I.T.
• Assignments in different organizational cultures:
  • Bureaucratic
  • Entrepreneurial (e.g. a start-up)
  • Lean
• Assignments using different development methodologies:
  • Agile (process-light)
  • RUP (process-heavy)
  • Waterfall
  • Chaotic
• Assignments working on different types of software systems:
  • Web applications
  • Enterprise systems (e.g. messaging, batch processing, integration of systems)
  • Desktop
  • Mobile
  • Games
  • Software development tools (e.g. compilers, IDEs)
  • Software infrastructure (e.g. operating systems, databases, networking)
  • Scientific (e.g. simulations)
  • Mission critical (e.g. medical, aviation)
• Assignments involving different software development skills:
  • Web design (html, CSS)
  • User interface design
  • Database design
  • Database administration
  • Data conversion
  • Web services
  • Batch processing

Obviously you cannot gain experience in all of the items listed above. The point is to think about where you want to take your career as a software developer and then choose assignments that provide sufficient breadth while avoiding irrelevant assignments.

Adopting Rotations

How can rotations for software developers be adopted? As much as I think it makes sense, I do not see the I.T. industry as a whole formally adopting rotations in a fashion similar to medicine. So what about at the level of an individual organization? Even fairly small companies usually have different teams or products for which rotations could be formally set up. I have seen companies doing this for undesirable work such as customer support or production support. Developers are rotated into such roles either for an entire iteration or for a certain percentage of days each month. But the focus of these rotations is typically on getting the undesirable work done rather than the increased learning.

In many companies I have seen a tendency for developers to get stuck into a particular role or team. Supervisors are reluctant to give up a trained, skilled resource to another team, even in a trade. Temporary assignments on another team are also fairly rare for the same reasons plus the common fear by the supervisor that the temporary situation ends up becoming permanent. So a developer who wants a change often needs to initiate it by changing jobs.

Since the industry as a whole as well as individual organizations cannot be relied upon to provide for rotations, I believe it becomes the responsibility of each individual software developer to seek out that breadth of experience. Formal rotations may be beyond our control, but we can explicitly seek out different assignments to provide that breadth of experience and accelerated learning.

I'm interested in hearing what you think. Do you agree with my list of required rotations? Are there any you would add or remove? Do you have examples of where you actually used rotations? Let me know by leaving a comment below.

The first experience was reading an article by Uncle Bob titled Master Craftsman Teams. Bob's main point is that the current model of developing expertise based on entering the field with a four year computer science degree and then haphazardly gaining experience is broken – dysfunctional and ineffective in both cost and time. The alternative Bob suggests is a formalized path of learning based on the apprentice – journeyman – master craftsman model used by the trades, but without starting with the degree.

The second experience was the recent birth of my second son which involved interacting with a variety of medical staff at the hospital. Like previous hospital visits, the staff included nurses and doctors in a variety of training roles – student nurses, interns, and residents – as well as the experienced staff in charge – running the ward or performing the surgery. These training roles are clearly defined with careful supervision. One student nurse we dealt with was under the direct, one-on-one supervision of an experienced nurse who was constantly teaching, training, and assessing the student. Medical interns are supervised by junior residents who are in turn supervised by senior residents with a full staff doctor ultimately responsible. Unlike the trades, medical professions do require a university degree. Practical experience, however, is heavily integrated into the degree program. After the degree formalized training is heavily integrated into the work: residents rotate between roles within their area of specialization and still have exams to take.

So how have these experiences influenced my thoughts? I agree with Uncle Bob that the current model of developing expertise is grossly deficient. Far too many times I have witnessed novice or junior developers being thrown onto a team and expected to churn out production code with limited (if any) supervision or review. I have also suffered multiple times through the experience of having a team member who made a negative contribution to the team: we would have been more productive without them.

Is there a better way? Definitely! I'll start with the university degree. I disagree with Uncle Bob in that I do feel that a university degree can be a useful foundation for a career as a software developer. I think that the classic computer science curriculum is not the best approach: I would rather see a stronger emphasis on practical software development topics and a corresponding reduction in highly theoretical courses. In other words, take more of an applied science or engineering approach rather than a pure / theoretical science stance. Course work should include programming assignments. This may sound funny to say, but from personal experience in school I know individuals that did not have to write any code until their third year of school. There should be at least a few project courses that focus on having small teams of students develop a piece of software end-to-end, starting with gathering requirements and culminating in a tested, working version. The undergraduate program I went through offered one such course (called Software Engineering) that I found beneficial. It would have been much more effective, however, with an experienced software developer to act as a mentor throughout the course.

More important than the curriculum, however, is the need to integrate actual work experience into the degree program. I am familiar with computer science degrees that offer an optional 16-month internship. This is better than nothing, but a single big work block is, I believe, the least effective way to integrate work with school. In my undergraduate degree in computer engineering I went through a co-op program that after the second year alternated one to two terms (four-month blocks) of school with one to two terms of work, including summers. I purposely changed employers each work term to maximize the variety of experiences that I encountered which I believe this greatly enhanced my learning.

As beneficial as I found the co-op program, I know that it could be made far more effective. Work terms are actual paid job placements so the employers are motivated to get their money's worth from the co-op students rather than provide formalized instruction & mentoring. In some cases students are dumped into a team that is too busy as it is to provide effective guidance to the student. In other cases students end up doing activities unrelated to software development. Let's compare this to the medical model where students go through rotations that explicitly target different areas of medicine with much tighter integration between practical experience and classroom training.

I think computer science degrees would benefit from providing structured, part-time work experience concurrent with part-time classes – perhaps working three days a week and in classes two days a week. In order for structured mentoring / training to be provided within the work setting, the only viable solution I see is for the student to work for free and have the university pay the supervisors an instructor fee. Four-month paid work terms, most commonly over the summer, could still be included in the program.

The following table outlines a sample degree program incorporating the suggestions I have made above:

I haven't discussed the topic of developing expertise in the work place after university – this post is long enough and I'll save it for later – but I will say that I do like Uncle Bob's craftsman-based model.

What changes do you think should be made to computer science degree programs? Let me know via a comment below.

• Myth #1: There is a 'magic' solution or technology that will solve all our problems. Sharon & Joni called this silver bullet thinking, which is a nod to the classic article "No Silver Bullets" by Fred Brooks, who argued that much of software developer involves essential complexity that no tool or practice can magically simplify or eliminate. Both technical and non-technical people can subscribe to this myth. Technical people should know better, but the optimists and early adopters are often suckers for the next new technology that comes along, believing that it will solve all the problems with the current approach. Good examples of this were Ruby on Rails for developing web applications, and now currently Erlang for doing concurrent processing. It is easier for non-technical business people and managers to buy into this myth because of their lack of technical knowledge, especially if the I.T. department or sales people are eagerly pitching a solution that they claim is the solution. Recently I helped out a software development team struggling with multiple parallel development activities on the same code base. They were already using branches and merging within their version control tool, so I was surprised to discover that the non-technical lead of this team was convinced that there was some feature or approach to using the version control tool that would magically make all their problems go away.
• Myth #2: If an I.T. project is properly managed and is successful – on time and budget – then the benefits will occur automatically. In order to be funded, most I.T. projects require a business case with a cost-benefit analysis that specifies the expected benefits from the project. Once the project is approved the business case is often forgotten about, especially by the I.T. team. The assumption is made that the benefits will occur automatically.

The typical result of believing these myths is that the I.T. project team celebrates their successful project, but the client is left unhappy or angry since the original expected benefits are not realized. To protect themselves, I.T. has adopted various practices such as expectation management, contracts, and audit trails. These help protect the I.T. team, but do not improve the customer's happiness.

This is a problem for the business. The solution is for the business – not I.T. - to ensure that business value is realized. I considered this a key insight of the presentation. The business owner must manage a business initiative potentially spanning multiple I.T. projects to ensure that the desired outcomes are achieved. Sharon and Joni stated that in their experience it usually takes up to five years after the completion of the first I.T. project before the appropriate level of benefits are realized. This seemed like an overly long period of time to me, and I wonder if their experience being with the government rather than corporations was a factor.

Sharon and Joni outlined a management framework called Value Management that helps business managers determine, evaluate, and finally achieve business value for an initiative. They called it a formal discipline similar to project management. The main insights behind this process are that expected business value must be defined up front along with measurable metrics as to whether it has been achieved, and then the initiative must be regularly evaluated and adjustments made as necessary to ensure that this value is realized. Sharon and Joni presented a number of examples within the City of Edmonton where the use of Value Management resulted in positive outcomes.

Sharon and Joni pointed out that organizations and business managers that use this type of process are silver bullet proof: rather than believing that technology will magically solve their problems, they should instead perform a disciplined evaluation of what they want to achieve and then manage to this outcome.

I think this is an important lesson for those of us in I.T. Not only should we be wary of silver-bullet thinking within ourselves, but we should also be aware that delivering business value is the ultimate goal of every I.T. project. Losing sight of that risks losing happy clients.

ITIL - Information Technology Infrastructure Library - is a library of books that defines a framework of processes for IT and provides guidance regarding their implementation. ITIL was created by the British government in the 1980's to help manage the IT organizations of its agencies. ITIL has continued to be maintained and updated by the British agency Office of Government Commerce (OGC), and the use of ITIL has spread across Europe and more recently North America. ITIL has also expanded beyond a set of books. ITIL was used as the basis for the recently-created ISO 20000 standard. ITIL also provides three levels of training and certification: Foundations, Practitioner, and Service Manager. The course I took was at the introductory Foundations level which focuses on concepts and terminology, leaving the details concerning each practice to be covered at the Practitioner level.

ITIL views IT as a collection of services. The provision of each service includes the necessary hardware, software, documentation, support, and management. This is a holistic viewpoint that aligns with the needs of the business, rather than organizing by technology which tends to result in separate silos that have difficulty coordinating. IT Service Management comprises the core of ITIL, focusing on the provision of high-quality IT services to the business in a cost-effective manner. Service Management is divided into two core areas, each consisting of a set of related processes: Service Support and Service Delivery.

Service Support focuses on the daily operation and support of IT services and consists of:

• Service Desk: Formerly known as the help desk, the service desk functions as the single point of contact for users.
• Incident Management: Handles incidents - interruptions to a service - with the goal of restoring normal operation as quickly as possible.
• Problem Management: Focuses on long-term improvements to reliability by determining the root cause of incidents and proactively preventing problems.
• Configuration Management: Provides a logical model of the IT infrastructure by tracking the status of all significant configuration items such as hardware and software. This model not only lists individual components, but also relationships between them. This is not the same as version control for application development (i.e. with version control software such as Subversion), but is at a higher level and is much broader in scope, which confused me initially. As an example, consider several applications deployed to several clusters of production servers. Configuration Management might decide to track each physical server and each application. For each application, the version deployed in production is identified. The list of servers that each application is deployed to could also be tracked.
• Change Management: Controls changes to the production environment by reviewing and approving or rejecting all changes and coordinating the implementation of the change.
• Release Management: Responsible for the building, packaging, testing, and deployment of a release - a collection of authorized changes. This can include deploying new software, new hardware, or both. I and others in the course were confused at the distinction between Change Management and Release Management. My impression is that Release Management is better named Release Packaging and Deployment because the role has very little to do with actual management and is subordinate to the Change Management role.

Service Delivery performs the long-term planning and improvement of IT services, operating at a tactical level rather than the daily operational level of the Service Support processes. Service Delivery consists of:

• Service Level Management: Primary point of contact with the customer - the payer of the bills - to ensure business needs are being met by agreeing to service level agreements (SLAs) and monitoring and reviewing the actual service levels achieved.
• Availability Management: Ensures the availability of the service according to the agreed service levels. This includes reliability, maintainability, serviceability, and security.
• Capacity Management: Ensures that both current and future capacity needs are provided cost effectively. Capacity includes both storage and performance requirements.
• Financial Management: Performs budgeting and accounting to explain the cost of providing each service to ensure that services are run cost effectively.
• Continuity Management: Ensures that IT services can be recovered within agreed time spans after a disaster, euphemistically called an "interruption to the business" by ITIL.

I found the Service Support area more relevant than Service Delivery given my time spent on application maintenance and support. The Service Support processes were an unexpected hybrid of practical, common-sense processes and theoretical or idealistic approaches. The essence of the Service Desk, Incident Management, and Problem Management are all common-sense and implemented in to some degree in many IT organizations, although I think more emphasis needs to be placed on Problem Management - both the root cause analysis and the proactive prevention of problems. Configuration Management was the most theoretical of the approaches. ITIL showed almost every other process workflow updating the database of configuration items maintained by Configuration Management. This may be valid at an abstract, conceptual level, but does not correspond at all to anything I have seen in practice. This made it hard to understand how Configuration Management would be applied. This was actually my biggest disappointment with the course - it did not cover the details on how to set up and execute each of the processes. As a result, I finished the course feeling that I had nothing concrete to apply to my day-to-day activities. When I talked to the instructor about this, he explained that process details are not covered in the introductory Foundations course I took, but they are addressed in the Practitioner level courses.

ITIL needs to be approached from a pragmatic viewpoint. The ITIL materials acknowledge that you do not need to implement all the processes immediately, and should only apply them to the extent that they add value versus imposing unnecessary overhead. There is a definite risk with ITIL that rule makers will want to rigorously implement all the processes beyond what is reasonable.

Should you pursue ITIL Service Management training or certification? For the typical software developer, I think the benefits are low. Certification only matters if clients or employers are asking for it. From what I have heard ITIL is growing in popularity within large organizations, so ITIL certification may grow in importance for consultants. The ISO 20000 standard that is based on ITIL is too new yet to determine if organizations in North America will adopt it. Ignoring certifications and standards, the ITIL Service Management Foundations material does provide some value. For those new to IT - whether they are managers or support staff, the material provides a good overview of IT as a whole. The most significant benefit of ITIL is that it provides a common language with which to discuss IT.

People I have dealt with professionally who are not software developers occasionally make comments that imply that writing enterprise business software is a fairly straight-forward task, easily handled by the average developer. I have challenged these statements by pointing out that reality disagrees. The majority of software development projects are overrun or canceled. Software that does make it into production often has serious operational failures, performance issues, or fails to meet users' requirements, and much of it is poorly designed, making it difficult to correct these issues and implement enhancements. These problems cannot be blamed solely upon weak developers. Across the industry, the typical software development team will be staffed by average developers (by definition), which means that the problems I just described affecting a majority of projects result from the efforts of average developers.

In defense of software developers, the leading causes of project failure or overrun are actually in the areas of project management (i.e. overly aggressive schedules) and requirements gathering. Problems in these areas cause much more serious impacts, and this is perhaps why project managers and business analysts sometimes feel that writing the code is the easy part. But it is not. Even assuming sufficient time, clear requirements, and a decent solution architecture (which are big assumptions), producing working, high-quality code is hard.

Why is this? I'll first look at meeting the functional requirements. For a typical enterprise business application many of the functional requirements are admittedly simple, and can be summarized as creating data entry screens for a particular set of data. Many of the enterprise business applications I have been involved with, however, also have more complex functional requirements such as complex business rules, batch processing, web services, or data conversion. These more challenging requirements are the ones most likely to cause problems for the development team.

Software must also meet non-functional requirements in areas such as performance, usability, and maintainability, and I believe that most enterprise software does poorly in these areas. It is simple to explain why: everyone is focused on meeting the functional requirements - clients, testers, and project managers as well as developers - so the non-functional requirements are neglected. Even when someone - most often the architect - considers the non-functional requirements, it is difficult to address them all at the same time. Typically multiple iterations are necessary. For example, a developer first writes code that meets the functional requirements, then tries out the user interface and make improvements to address usability issues, then tests and addresses performance issues, and finally cleans up the code to ensure the design is maintainable - easy to understand and modify. These multiple iterations take time, skill and dedication, and if a developer is lacking in one of these areas, the non-functional requirements will likely be neglected.

Combine the challenges in both the functional and non-functional requirements, and the result is that the typical software application will fall short in at least one area. Add in other common challenges like a compressed schedule and unclear or changing requirements, and the chance of success plummets even lower. In other words, producing good software is hard.

• Help software developers learn and grow as professionals.
• Increase the level of professionalism in the IT industry.
• Make the IT industry a more enjoyable and rewarding field to work in.

Why did I create a vision statement for my website? When I started my website I created a list of goals, most of which haven't actually changed. As I worked on this site, I came to realize that my motivation and passion for it are based on something more fundamental than what was expressed by these goals. After reflecting on this, I developed the three statements in my vision above. Looking at my past articles, I see that many if not all of them do speak directly to my vision, so I know is an accurate reflection of myself.

It was a surprisingly difficult process to write this vision statement. It may appear simple to you, but I put a lot of thought into those three sentences above. I wanted to truly capture my passion and motivation and express it in as succinct a form as possible. I had originally aimed for a single sentence, but I had to settle for three. As I came to the end of this process, I realized that while my vision was very meaningful to myself, it may not be as meaningful to others. So part of the purpose of this article is to more fully explain my vision.

My vision as stated above isn't actually limited just to this website but applies to everything I do related to IT: specifically, my full-time job as a software developer. Only a few of you reading my website have worked with me, but I hope that those of you who have will agree that my daily activities and attitude have reflected my vision. This vision of mine is in large part a vision of what software development and the IT industry in general has the potential to be. Some specific examples of my vision for IT are:

• Producing software that works, meets users' requirements, and is usable - software that delights the users and that is actually used.
• Carefully designed and coded software that is properly evolved over time, instead of the typical big ball of mud.
• Realistic schedules and meaningful deadlines.
• A professional work environment. No more extended, unpaid (and unproductive) overtime.

I know what I describe is an ideal far removed from current reality. But if you just accept the status-quo, then you have doomed yourself to remaining in it. Envisioning small improvements might be more realistic, but isn't very inspiring. My vision for IT motivates me and reflects my passion because it does aim so high. I can accept falling short of my vision due to reality, but I can't accept not trying to achieve it.