Agile Governance: Oil and Water?

When I first sat down to write about agile governance, my mixed metaphor and oxymoron meter started redlining. As I researched to see how prominent this topic has been, I encountered phrases like “tail wagging dog” and “are you kidding me?” It seems that the Agile advocates and the governance advocates don’t see eye to eye, if they even talk to each other at all. While several authors have addressed this topic (notably Dr. Peter Merrick in a recent article in Information Age), most of the coverage is theoretical. My goal in this post is to outline some practical approaches to ensure that the requirements of large-project governance can be met by engineering teams that employ Agile software development methodologies.

Let’s start with some working definitions. I define governance as accountability and risk management of IT projects and resources as perceived by non-engineering personnel. Agile can be defined as the set of engineering methodologies that rely on incremental development cycles, flexible long-term planning, and rigid short term-planning. Though many will comment on these definitions, they will serve for our purposes here.

Looking first at governance, it should not surprise anyone that the CFO or CEO of a Fortune 500 company might be interested in exactly how their $100M annual IT budget is being spent and how this spending benefits the company. That’s their job as representatives of the business interests of the organization. Similarly, the business stakeholders at an independent software vendor, such as account representatives and product managers, need visibility into the activities in engineering in order to communicate to customers, investors and other external stakeholders. Put another way, governance of engineering assets is a requirement for business – without it, the engineering or IT organization is operating in an opaque fashion without accountability.

On the agile side, it should also not be a surprise that after decades of being blamed for late projects, over budget software releases, and failure to meet customer requirements, engineering organizations are fast embracing agile as a way to manage software development projects. Starting with the axiom that requirements are rarely fully specified, agile provides a scheduling and requirements management framework that enables teams to work successfully on under-specified requirements today, and adjust their plans as requirements change tomorrow, next week, or next month. Put another way, agile solves the requirements management problem that every business has – without this solution, the engineering or IT organization may be transparent (“See, here’s our project plan”), but is likely not satisfying business requirements.

Adrift in a Sea of Conflicting Priorities and Assignments? Here’s a Life Preserver!

Do you ever feel like things are out of control on your project, that you are adrift in a sea of conflicting priorities and requests? Do you suddenly find out at the last minute that you are the bottleneck and everybody is breathing down your neck asking you what is taking you so long to create the moveStuff() method but you had no idea that anybody even cared about moveStuff() or that you owned it? Do you ever find yourself in the exact opposite position, wondering why Sue and Bob didn’t get their stuff done that you need and then your boss walks by while you are surfing the net waiting for Sue and Bob? And who is Bob anyway?

The solution is simple! All you need to do is get everybody to move to Project. Well, if you have somebody you can spare full-time to keep Project up to date of course. Oh and I almost forgot, you’ll need to start using a requirements tool. But that’s it really, other than integrating them all together over the weekend and of course that’s assuming you’ve already gotten a CRM tool for workflow.

There is a simpler solution. It isn’t perfect, and it doesn’t solve all problems, but it can definitely provide the following benefits:

• reduce the chaos
• increase your vision into where you are and what’s going on
• reduce the number of status and/or project management meetings
• reduce the need to provide the same information over and over again
• simplify collaboration both locally and for distributed teams
• provide a more Agile workflow

The answer is to reduce the amount of rework that you are already doing. Right now you are probably storing defects in a defect tracking system, enhancements (aka RFEs, requirements, etc) in a requirements management tool (usually Excel or Word but sometimes an actual RM tool), and if you are using a project management tool it is probably MS-Project. All three of these product areas evolved to provide different aspects of project management for different groups of people and as a result they have lots of overlap. Considering how hard it is to coordinate three different systems, why not consider standardizing on one system for most of the work? The only question is, which system?

If we are going to try to do most of our project management work in a single tool, we should first decide what the interesting activities are. I believe they are: recording enhancement requests and defects as they are gathered by marketing or reported by users, load balancing, estimated completion calculation, critical path determination, work assignment, workflow, and reporting.

First let’s consider how well suited Project is for doing most or all of these tasks. Project is good at taking a small static collection of large tasks and doing load balancing, estimated completion, and critical path determination. Thus, it is mostly used for the very narrow task of project management of enhancements.

Next let’s consider requirements management. For whatever reason, most people use Excel or Word as their requirements management tool instead of a “real” requirements management tool. Excel and Word are just not appropriate for project management.

Lastly, there is defect tracking. A defect tracking system covers the assignment, tracking, workflow and reporting of defects. There is usually a higher volume of defects than enhancements, and they are usually smaller in scope and have a more complicated and often more time critical workflow. If it works well for defects, it should work equally well for enhancements.

Based on this analysis, it makes sense to extend the project management that you are already doing with a defect tracking system to include enhancements. A generic name for something that is either a requirement, enhancement, or defect is “work item.” By using work items to track all work, it is easy to see where you are and what remains to be done. Now you can use a similar workflow for enhancements as you do for defects, for instance from newly entered, to triaging, to assignment, to test development, to completion, to test, to integration, to delivery. You can easily run a query to see which work items have their code written but do not yet have any tests. Similarly, you can see which work items are done from a coding perspective and have tests but have not yet been verified as done by QA. This will give you a much more complete view of your overall project status and progress.

Whatever you are currently using for defect tracking it will be straightforward to start getting the benefits of managing defects and enhancements together. Just add a field that indicates if a work item is a defect or an enhancement. You may need to make a few more changes to accommodate a slightly different workflow for enhancements than you have for defects, but I think you’ll find it is worth the effort. For one example of how this can work, you can take a look at how AccuRev does it using AccuWorkflow.