Pattern for Private Prototype Development

by Dave Thomas

Your team has been coding a feature for a few days, across dozens of files, and everyone is excited with progress.  Deep into development, you find an interesting 3rd party library that may simplify your work and possibly take the team’s feature to the next level.  But it will require adding new files, moving some directories, and refactoring some critical code.  However, you’re not ready to commit your current changes because they are unfinished and will surely break everyone else.  Your changes need to be saved before prototyping in case of a rollback but also be integrated with prototype development.  And what if the new library is a bust?  If you start co-mingling the library integration with unfinished code, the potential revert process will be a complete nightmare and waste of time.

Private Versioning. With AccuRev’s private workspace, you can always commit your changes early, often, and safely without sharing amongst your peers.  But in this case you have two logical development efforts, the 2nd effort depends on the first and both need commits to preserve evolving changes.   How do you keep them cleanly separated and continue to work on both in parallel?

Stream Inheritance.  AccuRev streams have an intriguing and very powerful feature called inheritance.   Similar to how an OO subclass implicitly inherits methods from parent and grandparent classes, a child stream implicitly inherits versions of files & directories from parent and grandparent streams.   Taking advantage of inheritance, we can use streams to independently manage logical changes and cleanly maintain change dependencies without physically co-mingling files.

The Pattern.  Prototyping changes without co-mingling files can be done by simply creating a series of ‘personal’ or ‘private’ development streams (though, they are just regular dynamic streams). This pattern will create a “Feature”, “MyDevelopment”, and “MyPrototype” stream sub-hierarchy.  See Picture.

click to enlarge

From your Integration stream (or equivalent), start by creating a “Feature” stream that will collect all changes from your entire team.  [See related blog, Stream-per-Task Pattern].  The majority of team members may have their workspaces directly from here.  Next, create a child stream called “MyDevelopment” that will track your personal ongoing development activity.  Finally, create a grandchild stream called “MyPrototype” to track changes that will be discarded or retained depending on the level of success.

The prototype development activity is committed, shareable, integrated, and yet cleanly segregated from both active feature and private development.

The “Feature” stream is the collection point of all in-progress development activities for the given feature from the entire team.  Developers will promote here frequently possibly kicking off an automated build with success/fail notification.   The “MyDevelopment” stream provides a collection point for all of your personal development changes.  This stream may be considered a “private” development stream simply because no other developers will likely use it – set a stream lock to be sure.  A lightweight Continuous Integration build (i.e. compilation only) may be performed on “MyDevelopment” for sanity sake or just compile and promote as a practice.  The “MyPrototype” stream is a collection point for all prototype changes. Even as new changes are promoted to “Feature” and “MyDevelopment”, the “MyPrototype” stream will automatically incorporate those changes (via inheritance) and the prototype developers will merge changes as necessary. The prototype development activity is committed, shareable, integrated, and yet cleanly segregated from both active feature and personal development. Also, by using a stream for prototype work, multiple developers can contribute and collaborate. If the prototyping work is deemed successful, the files can be promoted to “MyDevelopment”. If the prototyping efforts don’t work out – no problem – just remove the “MyPrototype” stream and re-purpose the workspaces.

The beauty of this pattern is that it isolates development activity by purpose without co-mingling physical file changes.   It lets the prototype developers go nuts and shoot from the hip while the regular feature developers (with the deadline!) work unimpeded and without fear of rampant changes — and everyone stays up-to-date.   And with no limit to stream depth, teams can perform prototyping efforts in parallel and/or perform prototyping based on existing prototypes by adding another child stream!  Furthermore, the pattern works for any size development activity or team, even for us Team-of-One developers with tons of ideas, fast fingers, and a few green screen x-terms (2 space, 80-char wrapping of course – <chuckle>).

This is a perfect example of how AccuRev empowers the developer to take control of their own development.  Creating streams is extremely easy and the stream browser provides unprecedented visibility into the entire development process.  With the right amount of security in place (Locks, ACLs, Triggers), the critical release streams (left side of the stream structure) can be locked down by the CM Admins, but the development related streams (right side) can be fair game for developers to create an environment that suits their purpose, such as prototyping.

Does anyone have a good story to tell about how this pattern (or equivalent) helped with a major refactoring effort or library upgrade?

/happy prototyping/ – dave

Understanding AccuRev Stream inheritance, the short version…

In my role working with prospects evaluating AccuRev – and other SCM systems – the following question comes up often; “How are streams different from branches?” What’s the big deal? Usually the question is raised in the early stages of investigation, before gaining a real understanding and appreciation of the superior architectural differences that AccuRev brings to the table. There are other more comprehensive SCM resources out there which you can use for education, but I thought I’d present a brief, and quite real customer example.

Let’s say that you are using a branch-and-label version control tool, doing mostly mainline development (since branching is discouraged by a surprisingly large number of vendors out there!), and a major project comes up. Your Graphical User Interface has fallen behind the times and you want to update it to current standards. The boss assigns a developer or a team to this project and says, “Go to it, and I don’t want to see or hear from you for 6 months until it’s done.” So the team does what they have to do, creates a branch off the current mainline, and happily codes away on their “island” for 6 months.

Fast forward those 6 months. The project is reviewed, the GUI looks fantastic, and the go-ahead is given to roll it in; they want to release the hot new look-and-feel with the next major release. Oh by the way, the next major is scheduled for next month. Well, this is when that GUI team looks around at each other, scratches their heads and says, “How in the hexadecimal are we supposed to merge 6 months of changes back into the mainline?!?!” Not to mention getting them tested against what’s already been under development all this time.

AccuRev streams, and inheritance specifically, to the rescue!

Mainline development is going on with the core development team working on their code, and promoting up toward release through an Integration environment, on to the QA folks, and finally approved for GA. Meanwhile, when the GUI project got underway they simply created a dynamic stream based off of the Integration area. The developer(s) now can work in a private project stream hierarchy and their changes, even when promoted to their parent stream, will *not* impact the mainline at all. That’s very nice, but the flip side is where it gets cool. Because of inheritance, any changes that the mainline team promotes to the Int stream or higher are automatically going to flow down to the GUI proj. This means that they will be able to maintain a constant integration with the latest and greatest, while still advancing their own project in isolation. When that 6 months is up and the green light is given, the only step left is to promote all the work from the GUI proj stream into the integration stream. Merging has already been done (and tested as well!)

For the longer version, please see the following Stream-Based Architecture for SCM white paper by Damon Poole.

Sound like a better approach? I think so. What kind of challenges are you currently facing in your own development regarding branch/merge scenarios?