Why Traceability Matters

In a recent blog post on CodeSqueeze (http://www.codesqueeze.com/the-blame-game-how-necessary-is-traceability/) the subject of traceability and its utility in software project management was discussed. This post is a response to the author’s points on traceability. While I agree with the author on some points (especially that traceability can be used proactively), I believe that traceability has much greater utility in the development process and in the management of software projects.

Traceability, both at the code and issue levels, provides you with fundamental process metrics. By removing these metrics, you lose objective insight into the state of the project. Using such metrics for a blame game may indeed be a sign of poor project management, but not having the metrics at your disposal is a sign that you are not controlling the process, and that puts your project at risk. I can’t imagine other engineering disciplines such as chemical engineering even having this conversation – you measure (almost) everything that can be measured, and then determine which metrics are most useful in improving the process. The best engineers on your team will welcome objective measurements into their code – like having the fastest time at the racetrack, doing well by a set a metrics that are not easily manipulated is something to be proud of, and something for more junior engineers to aspire to.

As for feature traceability, some of the comments on the blog point in the right direction – for example, that in regulated industries, such traceability is a requirement. ‘We all trust each other’ doesn’t fly well with the FDA or DOD. But even beyond the regulated industries, traceability and related metrics answer questions that are too basic to leave unanswered:

– what requirement did this code satisfy?

– what ancillary issues (bugs, related requirements) is this code associated with?

– how long did it take compared to the initial estimate?

– how often since the fix was marked as ‘complete’ did the code change?

I could (and just might) write another blog post on why these metrics are interesting and essential to software process management, but for now I leave it to commenters to elaborate.

To the specific points that the author makes, lets look at them in turn. The author’s orginal points and counterpoints are in italics.

• Provides visibility of per person velocity – Visibility can occur during daily stand ups and weekly powerdowns.Sure it can, but that implies among other things that you are doing such standups. What if you are using an alternate process to coordinate the efforts of large, widely distributed teams? At some level, independent of your specific process, you need to be able to look into your SCM system and answer questions of who is doing what and why. Also, (and this touches on the second issue of accountability) not all developers communicate in the same way, so even if you are doing standups, as a manager you need objective facts to correlate with the verbal and written communications of your team.

• Creates a sense of accountability – Why the hell do you have people on your team that you don’t trust?  Trust doesn’t happen all at once. The basic principle of managing people, whether they are shipping clerks or software engineers is ‘trust but verify’. Once a person has established a track record of reliable communications that are consistent with the objective (and hopefully, automated) metrics, you might reduce the frequency of how often you check them against the metrics. But periodic checking is essential – if a previously reliable engineer has for some reason fallen off the tracks, they may be reluctant to admit it, or they may not even be aware of it. A properly constructed metric that is trending in the wrong direction will give you clue that there is an issue to be addressed with this engineer.

• Allows for possible learning opportunities – Either the bug is a bug (and no real lesson is to be learned), or senior developers did not properly guide less experienced devs through a design problem. All bugs teach you something. The latter case that the author mentions is certainly one lesson, but it implies that senior developers don’t write buggy code or are somehow immune to design problems (or design-to-implementation translation problems) of their own. This seems unlikely. Most of the best engineers that I’ve worked with will admit to some whopper bugs – many written well after they had earned the title of ‘senior engineer’ or ‘chief cook and bottlewasher’, or whatever. What a bug tells you is that you have code that needs attention, independent of whether it dropped out of a build process or was discovered by the janitor. The overhead for recording a bug, commenting on it and connecting it to the source code change that resolves it is minimal in a well-designed and well-tooled development process, so for a small amount of effort you get the benefit of a window into your code, its failings, and your process. Seems like a low price to pay for such potentially valuable information.

I’ll admit that traceability and metrics are on my mind often (see for example, my prior post on the AccuRev-Rally integration and how it assists in agile requirements traceability). Maybe it’s because my favorite toys as a kid were ruler, compass and slide rule and I just like measuring things for the heck of it. Or maybe it’s because I’ve seen too many engineering projects fail – projects that if the manager (myself included) had a bit less hubris and a few more objective facts at their disposal, could have been squarely in the success column.

 

Agile Requirements Traceability with AccuRev and Rally

Today, AccuRev and Rally announced our technology partnership. You can read the press release here. Since I was part of the engineering team that did the initial proof-of-concept integration between AccuRev and Rally, I thought I’d spend few moments writing about the integration and how it helps customers connect requirements managed in Rally with code changes managed in AccuRev.

First, for those of you who prefer the movie to the book, you can view a short video demonstration that I recorded here. It highlights the main functionality of the integration.

Now to the details. Rally provides agile project management, including story and defect tracking, to assist customers in managing the rapidly changing flow of requirements and issues that are common in Agile development processes. AccuRev provides innovative stream-based SCM and integrated issue tracking to enable software process automation within the SCM tool. The integration ties AccuRev and Rally together to tighten the loop between requirements and defects, and the code changes that developers perform in order to satisfy those requirements or fix the defects.

The integration, written in Ruby and Perl, consists of three parts.

1. The integration queries AccuWork, the integrated issue tracking system that is available with AccuRev, for all ‘New’ defects. It then transfers these defects into Rally, and makes an annotation in a custom Rally field called ‘AccuWork’. This field stores the AccuWork issue ID number.

2. When developers working in AccuRev make a code change, they will promote that code to an AccuWork issue, and place the ID of the Rally defect (created in the first part of the integration) in the promote comment. The AccuRev server_post_promote trigger (written in Perl) then fires and executes another piece of Ruby code that parses the promote transaction and sends relevant information about the code change to the Rally defect specified in the promote comment. This information is entered into Rally automaticaly and shows up as a ‘Discussion’ entry on the defect. Currently the integration posts the names and version identifiers of the AccuRev files that changed, the user id of the developer who did the change, and the timestamp of the change.

3. Finally, when a developer or Product Owner using Rally sees the code change, they can set the status of the defect to ‘Fixed’. Ruby code can then be run automatically that looks for all ‘Fixed’ issues in Rally, and changes their status to ‘Fixed’ in AccuWork. It does this by looking for the custom field ‘AccuWork’ created in the first part of the integration, extracting the issue ID number, and formulating an update XML message to AccuWork instructing AccuWork to update the specified issue.

That’s pretty much it. Working in Ruby and using Rally’s Ruby REST API was very straight-forward, as was working with the AccuRev XML API for querying and updating AccuWork. The end result is an early stage integration that provides basic requirements traceability between issues in AccuWork and Rally, and the coding activities of developers. I hope that if anyone is interested they’ll view the video and let us know what other features they’d like to see, so we can add them to our backlog and to future iterations of this integration in engineering.