The Land that Scrum Forgot

By Robert C. Martin

Scrum is a starting point. In fact, it’s a great starting point. But, as a framework rather than a full-blown methodology, Scrum is deliberately incomplete. Some things—such as the best technical practices to use—are left for individual teams to determine. This allows a team to create the best fit between their project and environment and an assortment of technical practices.

While that selection of practices should belong to the team or organization rather than to a group of methodologists, the benefits of some practices are becoming so compellingly obvious that they warrant consideration by any Scrum team. But, too many Scrum teams become complacent after achieving some early productivity gains with Scrum. And they stop seeking ways to improve. Many fail to try the technical practices necessary for long-term success. In the following article, Robert Martin (perhaps better known as "Uncle Bob") tells us why so many Scrum teams fail to sustain the promise of their early successes.

Continue reading this article.

An Example ScrumMaster's Checklist

By Michael James

An adequate ScrumMaster can handle two or three teams at a time. If you're content to limit your role to organizing meetings, enforcing timeboxes, and responding to the impediments people explicitly report, you can get by with part time attention to this role. The team will probably still exceed the baseline, pre-Scrum expectation at your organization, and probably nothing catastrophic will happen.

But if you can envision a team that has a great time accomplishing things you didn't previously consider possible, within a transformed organization -- consider being a great ScrumMaster.

A great ScrumMaster can handle one team at a time.

We recommend one dedicated ScrumMaster per team of about seven, especially when starting out.

If you haven't discovered all the work there is to do, tune in to your Product Owner, your team, your team's engineering practices, and the organization outside your team. While there's no single prescription, I've outlined some things I've seen ScrumMasters overlook.

Continue reading this article.

Effective Retrospectives & Reviews: A CSP’s perspective on continuously improving both the process and the product.

By Marco Mulder

Continuous improvement is a key principle of Scrum. Yet, though most Scrum teams conduct a sprint review and a retrospective at the end of each sprint, too many teams fail to implement the improvements they identify. In my experience, there are two common causes for this:

• Teams don’t have or maintain explicit standards for their product and process;

• Teams don't use the product backlog to schedule improvements.

Set a Dynamic Standard

A definition of done in Scrum defines general acceptance criteria to which all implemented features should adhere. This ensures that everyone has the same understanding of what it means for a feature to be “done.” Team members must all know these general acceptance criteria so that they know when they’re finished. The product owner and stakeholders need a definition of done so that they know what they can expect from the features delivered by the team. That does not mean, however, that the initial definition of done is the one you will want to use throughout the project's lifecycle. In fact, one of the reasons why an explicit definition of done is important is that it not only says what will be included in each feature, it also makes explicit what is not included. Some of the things not included may be planned additions later in the project.

Continue reading this article.

The top six technical practices every Product Owner must know about

By Mikael Boman

So, you have made it. You are the Product Owner of the company's flagship product. Now you will spend your days thinking about business value and customer satisfaction, and leave all that technical stuff to the development team - you never understood it anyway.

Unfortunately, you're not off the hook so easily. In order to correctly prioritize the product backlog, you need to be aware of some of the technical items that will need to be included there. You need to understand the value of these items and recognize the terms the development team throws at you. Or worse, if the developers do not throw these terms at you, you will need to demand that they start using the technical practices you deem most valuable to your product.

If you recognize yourself in these first two paragraphs, this article is for you. I will explain the most important software development practices that you should know about. You need to know them, so you can understand their value when the team asks to take time in a Sprint to work on some of them, or you might need to put demands on the teams to introduce one or more of them to make your product even better.

This collection is by no means complete, but it includes the things I have encountered the most often in my career when working with development teams and Product Owners. The list of practices is highly influenced by the Extreme Programming (XP) practices, but the practices are presented and grouped a bit differently to reflect my view on them.

Practice #1: Version control system

This is something that almost every project team already has in place - if they don't, your product is probably in big trouble. What a version control system (VCS) provides is a place to store all versions of all files in the project, which in turn gives you an easy way to go back in time to previous versions of files when the need arises. It is, to use another common term, a system for source code management (SCM).

A VCS gives you an opportunity to, among other things, maintain a current and future release in parallel and keep track of who has changed what in the project files. It also enables you to use continuous build (more on that below).

There are several systems for handling version control, and which one is the best for your organization is a question that you, the Product Owner, probably do not need to worry about. A few examples of systems that are being used are Subversion, CVS, ClearCase, Team Foundation Server and Git.

Practice #2: Continuous integration

If you have a version control system (VCS) in place, your team should consider taking this step. A continuous integration (CI) system takes the code that has been submitted to the VCS and automatically builds a new version of the product.

The value of doing this is that you always have the latest version of your product built and ready, which allows for (more or less) immediate testing, manual or automated. You also get very fast feedback on any build-stopping problems that have been introduced with the latest code. Often the system is set up to report, via email or some other more immediate form of feedback, to the submitter of new code if the build breaks. If you have several teams working in parallel on the same product, you should have them make every effort to have the continuous integration build the entire product from its subsystems.

Another value of having a CI-system is that it forces the team to create a fast and automated build process. You will no longer have to be in the situation where the team says it cannot release the product because "only Ron knows how to system, and he is out with a cold for the week" or "sure, we can build a new release for you to test, but you have to wait for three hours because of all the manual steps we have to take to build the product."

Finally you should look into automatically deploying the built product as well - most CI systems will have support for that.

There are several systems for doing continuous integration. A few examples are CruiseControl, Hudson and Team Foundation Server.

Practice #3: Automated testing

Manually testing the product usually takes a lot of time. In fact, for larger systems it is not uncommon for the testing to take longer than the development of new functionality. So to improve this common situation, we introduce automated tests.

Automated testing can be introduced at several levels of development. Closest to the code is unit testing. This is a technique where the developer writes a few lines of code that tests other pieces of the code. These tests can be written before writing the source code (called Test-Driven Development, TDD) or after writing the source code (unsurprisingly called Test-After Development).

We can also introduce automated smoke- and integration-tests. These are especially useful in conjunction with a continuous integration system - after building the product automatically you run a few quick tests to make sure the product seems to work - at least enough to start and be ready for more serious testing later.

Going further, it is of great value to introduce automated regression and acceptance tests. This is, in many cases, the most challenging thing to accomplish. But the payback can also be a lot bigger if you put in the initial effort of getting this in place. This is where most of the manual testing is usually done, and as mentioned earlier, it can take weeks and months to do this in a larger system. The regression testing is focused on ensuring the new code has not broken any old functionality, at least not any old functionality that is covered by the regression tests that are in place. The acceptance testing is done to ensure the product adheres to the requirements.

One major objection usually used by teams is that it is hard to automatically test graphical user interface (GUI) applications. It is to some extent true, but it is today quite doable in most modern software development environments. A couple of tools to have your team consider, if they are up to the challenge of introducing this, is Selenium and Marathon.

The biggest gain in quality is achieved if your team can manage to introduce automated tests at all levels. But introducing it at just one level is much better than having no automated tests at all.

Practice #4: Refactoring

Refactoring is the art of rewriting the source code so that it does exactly the same thing, but in a different way. This may sound a lot like waste, and something the developers would love to do for the fun of it, but is in fact a requirement for being able to work in an agile way with software development. You need to take some time to rewrite things that are already working, to accommodate for new functionality. The alternative would be to try and anticipate every possible functionality that may be introduced in the product, and this has been proven again and again to be a futile task - there will always be changes in the requirements while working on a product.

Refactoring also takes into account that the developers learn while building. So after building something, they will know a lot more than when they started, and can, with a little extra effort, make the code more maintainable, less error prone and more agile in the sense that new functionality can more easily be added to it.

To not continuously do refactoring, is to risk building up what is called technical debt. This means that if we just keep adding new functionality, sooner or later (most likely sooner), the team's progress will slow down due to technical limitations in the code. To be able to keep a steady, possibly increasing, pace of delivered functionality, the team needs to continuously work with refactoring.

To be able to do this without introducing a lot of bugs in existing functionality, automated testing (unit and/or regression) is almost a requirement. There is a lot of support for refactoring in today's development environments, but to do it efficiently is an art that the developers will need to learn. All you have to do, as the Product Owner, is to allow them some time to do this.

Practice #5: Simple design

Designing a software system entails deciding how different parts of the system should collaborate and how responsibilities should be divided between them. This can be done in (almost) as many ways as there are developers, and there is rarely only one correct choice. All too often the design ends up being the work of the smartest developer or architect on the team, using all the latest trends (oh yes, there are trends) in software design. Most often it is also a big up-front job, meaning the design of the whole system is done before the work of implementing any part of it has started. This leaves the more inexperienced developers more or less clueless about how things fit together, and they end up able to contribute less to future design discussions. Also, if/when the smartest developer, the one brain smart enough to comprehend the superb design of the system, leaves your company or moves on to a new product within your company, the rest of the developers are left feeling very confused and lost. They then start having a really hard time implementing new functionality within the framework of the design they do not fully understand.

To counter this problem, and to work more agile with the software development, we want to avoid doing a full design of the system up front, and instead let the design evolve while implementing functionalities of the product. To be able to do this, the team needs to keep the design simple. A simple design is more easily improved when the team learns more about the product they are building. A simple design is also easier to modify when requirements change mid-project. Additionally, a simple design is more likely to be understood by all members of the development team.

One technique that aids in making the design simple is to keep the domain model, the software implementation of the world your product will live in, defined in terminology that can be shared by everyone involved with the product - the developers, the domain experts, the Product Owner and the end users. This allows for much easier discussions - everyone can use the same terms for the same things. A lot can be learned from the area of Domain-Driven Design (DDD) about how to actually do this.

Practice #6: Collective code ownership

A quite common problem in software development is that without conscious effort, there will be parts of the code that no more than one of the developers dare touch for fear of breaking something. This makes the product much more vulnerable to that developer leaving the company or being on sick leave for an extended period. To avoid this, the team should make an effort to introduce collective code ownership.

In practice this means that there should be no parts of the code that only one developer knows. Every developer on the team cannot, in most instances, know exactly every part of the product, but all developers can be familiar with all of the code, and you should make sure to always have at least two developers that know more about each specific part. To accomplish this spreading of knowledge we can use several methods. One way is to have code design discussions and reviews at regular intervals, where developers present parts of the code to each other and discuss the chosen solution and possible refactorings. You can also make it a requirement for a developer to discuss a chosen solution with at least one other developer before it can be submitted to the version control system. A way of making the knowledge transfer automatic is to introduce pair programming. This entails two developers sitting at one computer, solving a problem together. This may sound like a waste of resources, but those who start using it often find that the quality of the code produced, and the reduced cost in time for knowledge transfer, more than make up for the time "lost" when two developers work on one task.

To make it easier for the team to actually do this, I recommend you have two things in place. The first thing is a common coding standard, to ensure developers write their code in the same way. It need not be extremely detailed, but it should at least have some commonality in how the code is structured regardless of which developer has coded it. The second thing is the simple design I discussed as practice number 5.

Final words

So now you know my top priority practices. My suggestion to you is to spend time and effort implementing them; if not all at once, start trying to implement at least a few of them. The aim of these practices is not to make the developers happier because they get to use them, which they very well might be - developers most often want to create better code - but rather to make your product better. ‘Better’ in this context meaning more maintainable and more easily changed when you have changes in your requirements. Best of luck to you!

The Hills Are Alive with the Sound of Scrum

By Paul Goddard

I am always on the look out for new and interesting metaphors for the use and benefits of Scrum specifically and an agile approach to delivery in general. As a keen and almost-able musician myself, the title of Mike Cohn’s recent article, “Leader of the Band,” got me thinking more about the parallels between agile and music: both in its composition and its performance.

Creation

I do not profess to be skilled in composition or arrangement of music. I can, however, imagine how a composer visualizes a piece of music before it has been completed. Before he has written a single note, a music composer must have an idea of the style, structure and mood of the piece he is about to create. He starts with that vision. So, too, must a Scrum project begin with a theme or picture of what ultimately will be delivered.

In music, this vision comes from many different sources. Compositions can be written to mark a specific event, reflect a particular emotion or experience, or to convey a particular mood in a play or movie. Historically, many classical compositions have been written to mark specific events. For example, Piotr Ilyitch Tchaikovsky’s 1812 Overture (1880) was composed to commemorate the unsuccessful French invasion of Russia in 1812, a major turning point in the Napoleonic Wars. Tchaikovsky’s clearly had the intention of building a classical masterpiece that portrayed the increasing magnitude and frenzy involved in the final stages of battle. Other pieces of classical music have been written to encapsulate a composer’s own thoughts, experiences or emotional state. The Symphony No. 9,  From the New World, popularly known as the New World Symphony, was composed by Antonín Dvořák in 1893 during his visit to the United States from 1892 to 1895. In an article published in the New York Herald (December 15th, 1893) Dvořák explained how Native American music had been an influence on his vision for this symphony:

"I have not actually used any of the [Native American] melodies. I have simply written original themes embodying the peculiarities of the Indian music, and, using these themes as subjects, have developed them with all the resources of modern rhythms, counterpoint, and orchestral colour."

- Antonín Dvořák

In some cases, composers are asked to write a movie soundtrack that encapsulates the themes of the motion picture. Good examples are the iconic works of John Williams (Star Wars, Jaws, Superman) and the more sedate film scores of James Horner (Titanic, Braveheart, Apollo 13). In these cases, the film director has a great deal of input as to which parts of the soundtrack are added to the film structure to improve the final product. He must work closely with the composer to ensure that his own vision is realized, in most cases working with only the movie’s script, rather than the visual elements of the motion picture. In all of these instances, a clear project vision gives structure to what the composer will create. The project has a purpose: deliver something that will make people feel, remember or understand a particular mood, experience or event.

In software development, the vision usually comes to the team from someone like a film director: a product owner. The product owner gives a clear picture of his goals and writes, or helps to write, stories that describe the things he’d like to have in the final product. Working collaboratively with the team throughout the project, he guides the team toward the creation of a product that fits his, or the organization’s, needs.

Playback

I also see the emergence and development of musical piece as a distinctly agile practice. Picture a composer sitting at a piano, armed only with blank manuscript and a vision. The ideas in the composer’s head are translated into sound using the piano. The sound provides instant feedback as to which melodies, harmonies and chords combine effectively; in other words, the composer is testing and integrating while the piece develops. In most cases, a musical creation starts small: a simple melody or sequence or chords from which the composer can choose to overlay harmonies and counter melodies. Through simple playback and constant rehearsal the composer can quickly identify problems or imperfections in the product. Working with this integrated product allows the composer to add depth to the music against a solid, tested foundation.

These simple common sense practices translate directly to the quality engineering practices that underpin a Scrum development project. A test-first approach always minimizes the risk that our software “music” will not integrate at the end of a sprint.

Reducing Risk

“To achieve great things, two things are needed; a plan, and not quite enough time.”

– Leonard Bernstein

Scrum allows us to reduce the risk of undelivered projects through iterative delivery and striving for a potentially shippable product increment at the end of every sprint. I found many examples from music that characterize a similar low-risk approach to composing. This is best seen from composers who repeat a simple melody throughout the piece, then add substance to the music by changing the mix of instruments or by increasing the volume or tempo. These changes move the one simple melody towards a dramatic crescendo.

This is most noticeable in pieces such as Pachelbel’s Canon in D Major (1680), Dukas’ Sorcerer’s Apprentice (1897) and Grieg’s In the Hall of the Mountain King (1876). It is also evident in more modern music, where artists build on a simple theme with instruments and vocals, while the original melody still remains identifiable. This is a relatively safe way of adding depth to music, one instrument or rhythm at a time. This can be recognised in the Rolling Stones’ "Sympathy for the Devil" (1968) and more recently in the dance anthem "Sunchyme" by Dario G (1997). When listening to these pieces, clear sections are audible where the composer has modified the melody only slightly while still adding value. Each section is its own marketable product, which could be released if necessary even if the entire piece has not been written. The risk in not having a completed composition has been greatly reduced.

In the same way, adding functionality to a software project in small chunks adds to its depth and usability, but at the end of any iteration, you are still left with a potentially shippable product, greatly reducing the risk that your software project will fail to deliver.

Continuous Integration

Modern music recording in studios allows musicians to record iteratively and constantly check the quality of the outcome. Components of the song or music are usually recorded in isolation, but need to be integrated effectively to form a quality final product. Digital recording now allows musicians to playback drums, bass or treble to allow vocals to be added and tested against each other. This kind of dedication to testing and feedback continuously throughout production ensures the product achieves the artist’s goals. The same can be said of a Scrum project.

Self-management

As a former member of a brass band, I can certainly relate to Mike Cohn’s observations on the close similarity between a ScrumMaster and a band/orchestral conductor. The conductor (or musical director) maximizes the performance by ensuring the musicians carry out the composer’s vision when playing the piece. Some conductors can become quite animated as a means of injecting passion into their ensemble. However, a conductor does not have the capacity, or in most cases the technical proficiency, to take the place of a team member.

A good conductor cannot make a poor band play great music. But a good conductor can point out and help bands tackle some of their biggest problems. He can work with the band to improve those aspects that can be improved and can find ways to maximize that which they do well. In the same way, Scrum teams benefit from a ScrumMaster who can keep them moving toward a project vision, removing impediments and emphasizing team strengths.

Ultimately, though, it is the band that makes the music and the software development team that creates the product. When I hear a band play great music, I am not applauding the conductor’s baton but the combined unity of the musicians making such a pleasant sound. In the same vein, when a band under performs, the audience regard it as a poor performance regardless of the conductor’s efforts to maximize the quality the band provides. Similarly, when a team delivers a great product (or a sub-par one), people notice the product, not the ScrumMaster.

“Life is a lot like jazz... its best when you improvise.”

– George Gerschwin

Although the conductor keeps the team on the straight and narrow, I have seen first hand how a band of musicians can continue to work effectively and still deliver in the absence of a conductor. If a conductor ceases to conduct, the music will continue, but there is more responsibility for musicians to listen to the person sitting next to them. Listening is one of the keys to self-management.

Similarly, for Scrum teams developing software, listening is a fundamental part of teamwork. During the daily scrum, we should spend about one minute talking and up to fourteen minutes listening. Teams who don’t listen to each other are going to struggle to work effectively as one team.

Another important factor for self-management is team size. In large bands or orchestras, instruments generally split into sections which have to function as a section to add value to the overall sound of the musical arrangement. In most cases, section leaders help lead these smaller groups if only to ensure the sections begin and end their parts in a smooth and together fashion. This is particularly necessary in a larger orchestra when the conductor may be addressing a different section of the ensemble.  In these large musical arrangements, sections also need to be more aware of other sections sounds in order to maintain the quality of the overall piece. My former musical director would often shout, “If you can’t hear the trombones, you’re playing too loud”.

The same principles should apply in large scale scrum project, especially where multiple Scrum teams are involved. Experience working with Scrum teams tells us that teams between five and nine in size are most effective. Each small team needs to be fully aware of what the other teams are contributing to minimize the risk of delivering a poor quality product, or possibly no product at all. Teams who are functioning well, but playing too loudly, could even be sub-optimizing the system as a whole.

In summary

The Scrum framework helps teams to achieve excellence in delivering products by suggesting very few rules backed by simple and common sense principles. These common sense principles apply to many things we see or hear in our everyday lives. Music is an art we have encountered and enjoyed all our lives, yet the time, effort and dedication to composing, writing, performing and recording is something we probably have little experience in or have paid little attention to. The principles and practices involved in making music are not that different from those we use to deliver Scrum projects. Clear product visioning helps define and drive a delivery while regular feedback and continuous testing and integration reduce the risk of un-deployable products. When I played the cornet in a brass band, I felt part of a team. The team struggled if certain players failed to turn up for practice. The team would have to self-organize to fill the gap. Effective teams are born of constant listening and reflection. This becomes even more important in larger teams, or teams-of-teams. Similar to a large orchestra playing a symphony, each section of a large software project will need to understand the role of other sections as well as their own to ensure the composer’s (or, in this case, the product owner’s) vision is achieved.