By Samir Bellouti
In 2008, I began work with a client on a new project. The client was a airline and travel agency that needed to rewrite, from scratch, their online travel booking application. The new website had the following primary requirements:
- Improve end user experience, including performance and security issues;
- Offer new set of products on the website (Hotel booking, car hiring and insurance purchase); and
- The new application had to connect to a brand new back-end system.
After learning about their process and project, I suggested that we try a new approach: Scrum. My clients did not know much about Scrum. In fact, the only Scrum-like practice my client had tried was daily meetings. I insisted that using Scrum could help us build software more quickly and build it with higher quality. This was not easy to sell. The client had a number of questions, such as:
- How can you accurately estimate a project with an iterative process?
- How can you determine the delivery date of a project if you re-estimate it after every sprint?
- How can your customer agree on an analysis and “sign it” if you do not have an analysis phase?
- Isn’t Scrum just a cowboy development process since we do not have a detailed design phase?
I answered their questions or worked with them to find answers. At the same time, I did not pretend that Scrum could solve all of their problems. I did point out what they already knew: their existing waterfall methodology, with its detailed estimates and phases, only gave the illusion that it could deliver a high quality product with all required features on budget and on time. Scrum, on the other hand, could mitigate some of these risks. After much debate, we decided to give Scrum a try.
Early Sprints
We called the first phase Sprint 0. We spent the iteration doing refactoring, as some of the code was written and the application was growing in size. We decided not to develop a user interface during this sprint but instead focus only on the framework and the design.
We quickly identified some problems with our first sprint. First, Sprint 0 had no pre-determined length. Instead, it lasted roughly two to three weeks. We decided that our next sprint would last four weeks--no more, no less. Second, we had not clearly defined our deliverables and goals at the beginning of the sprint. We learned that we should set our goals much more clearly for the subsequent sprints.
Even with our problems, we were highly encouraged by the outcome of Sprint 0. We managed to deliver most of our infrastructure components, including the globalization component, the interface to the Java-based SOAP legacy application, and the general structure of the web application.
By the end of Sprint 1, we had delivered our first functionality: the home page and a couple of related user stories. We had completed our sprint goals. We had developed solid product and sprint backlogs and had begun to use a tool to support us (Team Foundation). During this sprint, all the team members also had completed Scrum training, which helped them understand the Scrum philosophy and their responsibilities as Scrum team members.
By now the business was convinced that the team was capable of delivering working software every four weeks. They also began to believe that they could trust the team: they could give the team direction, let them work for four weeks, and only check the results at the end (rather than during the sprint). This realization was very important for the team and the organization. I believe that this is the most important thing in Scrum; trust the team. Trust the fact that the team can self-organize and deliver.
Immediate Effects on the Team
In most of the organizations I have worked for, when we talk about teams we actually mean a set of individuals grouped under the same manager. Scrum development teams are true teams. The difference is major.
On a waterfall “team,” the project manager plans the tasks and their precedence. Most of the time, this same Project Manager will then assign tasks to developers according to their skills, which are more or less well known by the PM. These tasks are generally large (generally taking few days if not few weeks to complete). In most cases, team members talk to each other only when there is an issue. Everyone on the team does his work on his own and, at the end, delivers what he was asked to do.
On a Scrum team, the team commits to deliver some set of functionality by the end of the sprint. No one person is accountable for a particular piece of the software; rather, the whole team is accountable for all the software they engaged to deliver by the end of the sprint. People choose their own tasks (or choose to work together on tasks). The tasks they pick from are relatively small (taking a few hours to a day to complete). At the beginning of each day, the team meets to talk about what they worked on yesterday, what they’ll be working on today, and what kind of help they need to complete their work. Throughout each day, Scrum teams reach out to each other for help or to coordinate tasks. At the end of the sprint, the whole team is accountable for what is delivered. There is no “I finished my tasks, so I’m good.” The sprint is not a success unless every promised feature is delivered.
As I mentioned earlier, the tasks in Scrum are relatively small. On our project, we limited the size to less than sixteen hours each. A detailed task list improves team productivity for the following reasons:
- A list of very specific tasks reduces or eliminates work on tangential tasks—those that may be important but are not part of the current sprint. A common example for our team was when a developer would grow interested in a performance or security issue and would interrupt his primary task to focus on these side issues. It’s not that these tasks are not important. They are. However, they often caused us to be late with the work we had committed to deliver. With Scrum, developers must only work on tasks from the backlog. If security or performance tasks do not appear in the list, no one should work on them. If anyone does choose to do work outside the backlog, he/she will have to justify it to the whole team during the daily meeting.
- By having a clearly defined task list, no one can say, “I don’t know what to do today,” and then wait for someone to assign him a task. On a Scrum team, a developer should pick up something from the task list and get to work, even if it’s not something he might normally think of as “his job.”
- Tracking small, specific tasks is easier that tracking high-level tasks. If a user story (use case) has multiple tasks and, for some reason cannot be finished, it is easy to find out exactly what is wrong (which task is slipping).
Working on a Scrum team means that team members will be asked to be accountable to each other. They will be asked to interact with each other daily. They may have to do work outside their comfort zone or help with tasks they wouldn’t normally undertake. This level of communication and coordination is not natural in IT world where developers are chosen for their autonomy. But the beauty of Scrum is that it really works. I can tell you that it took us less than two sprints (two months) to raise the level of trust among team members and improve the performance and the communication level of a team that had been working together for a couple of years.
The Effect on the Business
During Sprint 3, our software was growing from the functionality point of view. We were no longer increasing the number of lines of codes without seeing a concrete deliverable in return (concrete here refers to something the business or project owner can measure and play with).
At the end of Sprint 3, the business and the CIO of the company were convinced that this “new” methodology would help to deliver the software more quickly than with the waterfall model. The fact that the marketing guys could play with the application gave them the opportunity to point out a few potential issues. We prioritized these issues, bringing the most important into our next sprint and adding the less important ones to our product backlog.
During the Sprint 5 planning meeting we had a breakthrough. After hearing that we could not complete a feature by a certain date, our business sponsor reacted in a surprising way. Rather than bemoaning the loss she thanked us for the advance notice. “Usually I only find out a month before we’re going live that I’m not going to have a certain feature. This time, I know five months in advance. That gives me some options.” This was a very big step forward in terms of the relationship between the IT department and the business. They can finally see the impact of a change on the overall schedule. Moreover, they understand that they have to cut some other backlogged items in order to have their change done. This helps them to deal with the reality: a change has a cost and the development team has a limited capacity; either you cut somewhere else or you hire new developers. You cannot ask for changes and just hope that everything will be delivered on time.
For me, who insisted that the client to adopt Scrum, this statement was a real proof that iterative processes are the way to go for any software development project. I see, even today, many projects where development teams keep using the waterfall process. They all have their own reasons for avoiding a shift to a new approach. I strongly believe that the underlying objection is a combination of a fear of the unknown and a real lack of knowledge of Scrum. Most of the project managers I talk to refuse to admit that the waterfall model is over. Some other PMs, who use heavy iterative methodologies, will argue that it is too hard to trust a team for a whole sprint before they can see the results. That’s a very interesting point and a fundamental aspect of team and project management.
I know that in our case, after just a few months of using Scrum, we and the business had a much better understanding of our capacity (what we can deliver in a certain time frame). Also, because requirements, change requests and bugs were all logged in the PBI, everyone involved with the project had a more accurate view on what was involved in delivering the product and what remained to be done before the delivery date. Having this detailed backlog enabled the business to cut some nice-to- have functionalities in order to ensure the delivery of a critical functionality or bug fix. Because we knew our capacity and had a clear view of what remained to be done, we could predict fairly accurately what we could accomplish over the next 5 months.
One Year Later
After a year of sustained efforts, we delivered the first version of the application, a public website. Our last sprint was dedicated to bug fixing only. We expected that, as is the case with most multi-million-dollar projects, we’d be putting in massive amounts of overtime as we made the final push to go live. We did not have to work any overtime. None.
Why is that? We have some theories:
- Quality assurance (testing) was done as part of the sprints. Therefore most defects were already known, had been prioritized, and in most cases had already been fixed.
- Our capacity was well known and our estimates gained precision as the project went on. (We did find that we had to work to avoid the trap of overestimating tasks in order to make the sprints easier.)
- The business knew the application very well by the last sprint and we did not have major change requests. This helped us to stabilize the code and focus only on remaining bugs.
After the launch date, no priority 1 issues were reported. We also did not have any major complaints, which pleased us greatly as we were serving thousands of unique users a day and dozens of closed transactions. The iterative aspect of Scrum played a major role in this result. We addressed security, performance and functional issues long before the live date, which mitigated the impact of the changes we made.
From my experience, Scrum’s concepts are often viewed at first as an unreachable ideal. Self-organizing teams with facilitators in place of command-and-control managers sound great in theory, but organizations are predisposed to believe they won’t work in reality. What we found, though, was that Scrum’s common sense rules really do work, mainly because of their simplicity.
My advice is to try it before you dismiss it out of hand. We’re very glad we adopted it.
