Eclipse BIRT and AccuRev for Developer Reporting cont’d

In the prior blog on this topic I talked about the types of data I wanted to collect for developer reporting, and the basics I had to do to get the information out of the AccuRev software configuration management tool and into the Eclipse BIRT tool in Eclipse.

I promised charts, but before I get there, I realized I had to do a little more work to massage the data for use with BIRT. When you do an issue query in AccuRev (see prior post) the details for AccuRev users are only exported as the id number. To get this to look nice I ran a second command (accurev show -fx users) to get the user information as xml data. Most AccuRev commands have xml output in the format of <AcResponse>…</AcResponse> so I then ran this result through another xsl parser. My xsl file is straightforward and looks like this:

<?xml version=’1.0′ encoding=’utf-8′ ?>
<xsl:stylesheet xmlns:xsl=”http://www.w3.org/1999/XSL/Transform&#8221; version=”1.0″>
<xsl:output method=”xml”/>

<xsl:template match=”/”>
<users>
<xsl:apply-templates select=”/AcResponse”/>
</users>
</xsl:template>

<xsl:template match=”AcResponse”>
<xsl:apply-templates/>
</xsl:template>

<xsl:template match=”Element”>
<user>
<number>
<xsl:value-of select=”@Number”/>
</number>
<name>
<xsl:value-of select=”@Name”/>
</name>
</user>
</xsl:template>
</xsl:stylesheet>

This generated user tags which I could then use as a lookup when converting the issue data. The output of running this gave me the following xml file (names removed):

<?xml version=”1.0″ encoding=”UTF-8″?><users>
<user><number>1</number><name>…</name></user>
<user><number>2</number><name>…</name></user>
</users>

I could now run the issue xml through another xsl translation, which would use the xml above as a lookup table for user details. The issue xsl I used looks like this:

<?xml version=’1.0′ encoding=’utf-8′ ?>
<xsl:stylesheet xmlns:xsl=”http://www.w3.org/1999/XSL/Transform&#8221; version=”1.0″>
<xsl:output method=”xml”/>

<!– Begin Lookup table of users –>
<!– Setup key –>
<xsl:key name=”user-lookup” match=”user” use=”number”/>
<!– Setup variable for users lookup file –>
<xsl:variable name=”users-top” select=”document(‘activeUsers.out’)/users”/>

<!– Setup match for lookup of users –>
<xsl:template match=”users”>
<xsl:param name=”curr-user”/>
<xsl:value-of select=”key(‘user-lookup’, $curr-user)/name”/>
</xsl:template>
<!– End setup match for lookup of users –>
<!– End Lookup table of users –>

<!– Setup match for submittedBy –>
<xsl:template match=”submittedBy”>
<xsl:apply-templates select=”$users-top”>
<xsl:with-param name=”curr-user” select=”.”/>
</xsl:apply-templates>
</xsl:template>
<!– End match for submittedBy –>

<!– Setup match for assignedTo –>
<xsl:template match=”assignedTo”>
<xsl:apply-templates select=”$users-top”>
<xsl:with-param name=”curr-user” select=”.”/>
</xsl:apply-templates>
</xsl:template>
<!– End match for assignedTo –>

<!– Setup match for closedBy –>
<xsl:template match=”closedBy”>
<xsl:apply-templates select=”$users-top”>
<xsl:with-param name=”curr-user” select=”.”/>
</xsl:apply-templates>
</xsl:template>
<!– End match for closedBy –>

<xsl:template match=”/”>
<xsl:apply-templates select=”/acResponse”/>
</xsl:template>

<xsl:template match=”issues”>
<issues>
<xsl:apply-templates/>
</issues>
</xsl:template>

<xsl:template match=”issue”>
<issue>
<issueNum>
<xsl:value-of select=”issueNum/.”/>
</issueNum>
<status>
<xsl:apply-templates select=”status”/>
</status>
<state>
<xsl:apply-templates select=”state”/>
</state>
<severity>
<xsl:apply-templates select=”severity”/>
</severity>
<priority>
<xsl:value-of select=”priority/.”/>
</priority>
<dateSubmitted>
<xsl:value-of select=”dateSubmitted/.”/>
</dateSubmitted>
<dateClosed>
<xsl:value-of select=”dateClosed/.”/>
</dateClosed>
<dateCompleted>
<xsl:value-of select=”dateCompleted/.”/>
</dateCompleted>
<dateAssigned>
<xsl:value-of select=”dateAssigned/.”/>
</dateAssigned>
<submittedBy>
<xsl:apply-templates select=”submittedBy”/>
</submittedBy>
<assignedTo>
<xsl:apply-templates select=”assignedTo”/>
</assignedTo>
<closedBy>
<xsl:apply-templates select=”closedBy”/>
</closedBy>
</issue>
<!– xsl:apply-templates/ –>
</xsl:template>

<xsl:template match=”status”>
<xsl:attribute name=”name”>
<xsl:value-of select=”.”/>
</xsl:attribute>
<xsl:attribute name=”value”>
<xsl:choose>
<xsl:when test=”.=’New'”>
<xsl:value-of select=”‘1′”/>
</xsl:when>
<xsl:when test=”.=’Reviewed'”>
<xsl:value-of select=”‘2′”/>
</xsl:when>
<xsl:when test=”.=’Scheduled'”>
<xsl:value-of select=”‘3′”/>
</xsl:when>
<xsl:when test=”.=’Closed'”>
<xsl:value-of select=”‘4′”/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select=”‘0′”/>
</xsl:otherwise>
</xsl:choose>
</xsl:attribute>
</xsl:template>

<xsl:template match=”state”>
<xsl:attribute name=”name”>
<xsl:value-of select=”.”/>
</xsl:attribute>
<xsl:attribute name=”value”>
<xsl:choose>
<xsl:when test=”.=’Verified'”>
<xsl:value-of select=”‘3′”/>
</xsl:when>
<xsl:when test=”.=’Complete'”>
<xsl:value-of select=”‘4′”/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select=”‘0′”/>
</xsl:otherwise>
</xsl:choose>
</xsl:attribute>
</xsl:template>

<xsl:template match=”severity”>
<xsl:attribute name=”name”>
<xsl:value-of select=”.”/>
</xsl:attribute>
<xsl:attribute name=”value”>
<xsl:choose>
<xsl:when test=”.=’A'”>
<xsl:value-of select=”‘1′”/>
</xsl:when>
<xsl:when test=”.=’B'”>
<xsl:value-of select=”‘2′”/>
</xsl:when>
<xsl:when test=”.=’C'”>
<xsl:value-of select=”‘3′”/>
</xsl:when>
<xsl:when test=”.=’D'”>
<xsl:value-of select=”‘4′”/>
</xsl:when>
<xsl:when test=”.=’E'”>
<xsl:value-of select=”‘5′”/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select=”‘0′”/>
</xsl:otherwise>
</xsl:choose>
</xsl:attribute>
</xsl:template>
</xsl:stylesheet>

I had to scratch my head how to setup a lookup xml file, until I found this article on the IBM developerWorks site. It took some experimenting, but the xsl file above did the trick. I could have done a similar lookup for the status/state changes that are also in this xsl file, but it was quicker to enter these values directly. The side effect to keep in mind is because I don’t dynamically(today) get these values then if they change the xsl file needs to change.

After using this xsl file I get my new issue xml which looks like this (names removed and formatted for display):

<?xml version=”1.0″ encoding=”UTF-8″?>
<issues>
<issue>
<issueNum>1092</issueNum>
<status name=”Closed” value=”4″/>
<state name=”Verified” value=”3″/>
<severity name=”D” value=”4″/>
<priority>0</priority>
<dateSubmitted>943077600</dateSubmitted>
<dateClosed>1188926921</dateClosed>
<dateCompleted>1172186181</dateCompleted>
<dateAssigned>943077600</dateAssigned>
<submittedBy>…</submittedBy>
<assignedTo>…</assignedTo>
<closedBy>…</closedBy>
</issue>
<issue>
<issueNum>2614</issueNum>
<status name=”Scheduled” value=”3″/>
<state name=”Complete” value=”4″/>
<severity name=”C” value=”3″/>
<priority>0</priority>
<dateSubmitted>1007048693</dateSubmitted>
<dateClosed>1149106741</dateClosed>
<dateCompleted>1189024691</dateCompleted>
<dateAssigned>1120772200</dateAssigned>
<submittedBy>…</submittedBy>
<assignedTo/>
<closedBy>…</closedBy>
</issue>
</issues>

Phew. That being done, I now had a fairly automated method for extracting project issues, lookup tables for extraneous information, and xsl files to massage the data before processing in BIRT. Now I could bring the data into BIRT and begin setting up charts.

I spoke to some extent in the previous blog about the calculated fields I used in BIRT. I’ve since added a few more. I added a computed column which combines status and state, to make it easier to create certain chart groupings. Because these are two string fields it was as simple as putting in row[“status”]+”/”+row[“state”] as the expression.

I then also added the days to complete an issue, using this expression: (row[“dateCompleted”]-row[“dateSubmitted”])/86400

I now had enough information to create the 7 charts I wanted. I’ll highlight two that worked well for my purposes.

For completed issues by developer, I used the ‘assignedTo’ row as the Category (X) series, turned on grouping and set the interval to 0 so that each developer would get their own line. The Value (Y) Series was then the ‘issueNum’ row. From this I got the following chart:

Not bad, and it served my purpose. I could get a quick overview of how the issues were distributed. Some developers may cringe at seeing this type of information, as they fear they are not doing as much as their coworkers, but those reviewing the information have to keep in mind that there are lots of reasons for the issue distribution that is seen. Developers may be assigned more difficult work or not be on the project full time, and these basic numbers will not reflect the extent of these external factors. If you find you cannot easily answer these questions, then you need to find a way to track these external factors and include them in your issue management system.

For Severity Distribution, I wanted to take a look at what the general types of issues were that were resolved in the release. This was pretty easy. I created a pie chart, using issueNum as the series and severity as the category (grouping as text and getting the count). Using this I quickly got this chart:

Not a bad chart. Our use of severity A is in indication of crashes, so while some were fixed hopefully that is not the focus of the release. By the same token severity D and E cover a lot of cosmetic issues, so this release was a good distribution of the type of issues that should be addressed when preparing a release.

That’s about it for now. We’ve started adopting more agile practices, and in particular scrum methodologies, so now I want to revisit the BIRT configuration and start modifying it to generate burn down charts. Perhaps when I’ve made some progress here I’ll share some more.