Should we choose Iterative or Agile?

One of my current clients, a large government agency, have recognised that their current monothilitic waterfall approach doesn’t work all that well and are trying to decide whether to change their delivery approach to Agile or “just” Iterative (mini-waterfall style). 
 

Management have recognised that introducing Agile is a major change and that it requires a change in the way people go about their daily work, a different management style and a change in attitude and behaviour for everyone involved. 
 
I respect my client for recognising the magnitude of the change and I also appreciate that the attraction of an iterative development approach is that it doesn’t require much change. However, I think this is a “weasel-out solution” which, while helping management to hedge its bets, will not solve any of the fundamental problems associated with waterfall projects and will cheat my client of the benefits Agile could provide.
 
Here are 6 reasons why I would highly recommend to choose Agile rather than Iterative:

 

Iterative vs. Agile:

First of all, Agile is iterative already but it is way more than just iterative. Here are a number of differences between Agile and “just” Iterative development:
 

Mini-waterfall is still waterfall

Iterative is still waterfall, just on a smaller scale. A series of mini-waterfalls is certainly better and less risky than one big waterfall but mini-waterfall still is fundamentally waterfall and comes with all its known problems such as difficulty to adapt to change (“Nice idea but sorry, the requirements have been signed off months ago”), cascading delays (“Oops, we need to shorten the testing phase”) and low quality (“We don’t have time to fix those bugs. We’ll fix them in a later phase/iteration”). 

 

Agile is iterative AND incremental

Iterative development splits large problems into smaller chunks but without an incremental approach it requires a perfectly shaped idea and solution upfront as well as dead accurate estimation to build the right thing according to plan.
 
Agile assumes that the first idea is not necessarily the best and final one and that learning happens continuously throughout the process. It encourages a tight feedback loop to incorporate new knowledge, emerging requirements and innovative ideas even late in the process. Incrementing in addition to iterating towards a solution hugely increases the chances of success. 

 

Agile helps “building the right thing” through customer involvement

Iterative development does not address the lack of business and end user involvement usually seen in waterfall projects. Reports and PIRs time and again show that lack of business and end user involvement results in solving the wrong problem, making the wrong tradeoffs and increasing risk by not building core functionality first. 

Agile bakes customer involvement in through making the customer and end-user representative part of the delivery team throughout the entire duration of the project or life cycle of the product. Agile teams don’t second guess; they ask, watch and learn.

 

Agile delivers high quality

Iterative development still puts the testing phase at the end of each iteration. Testing at the end of an iteration results in defects not being detected until late in the iteration which usually makes it impossible to find the time to fix them. In addition, if any of the previous phases of analysis, design or build is delayed the test phase will be shortened and quality issues will go undetected even longer (Ever been on a project where bugs were found during systest or UAT?). 
 
Agile on the other hand does not have phases during an iteration and ensures testing is done continuously from the very beginning. Any bugs found during the iteration will be fixed before new work is started. This guarantees that we have working software at the end of each iteration with no work left to be done (no 90% done or known defects). Should we run out of time we compromise scope but never quality.

 

Agile is open to incorporating feedback and learnings

Iterative development has no mechanism built in to accommodate feedback and learning. Most of the time planning and design still happen upfront and scope is usually well-defined and fixed. Change arising from user testing, stakeholder involvement, increased understanding of the problem and an ever changing world can’t easily be incorporated into a just iterative approach without increasing project risk. Also, iterations can be as long as three to six months long which makes collecting feedback an infrequent matter and increases the risk of being out of touch with customer and business needs. 
 
Agile limits iteration length to one to four weeks which ensures that feedback is received in a timely manner and can be used to adjust course. Scope is somewhat variable and defined in user and business goals which makes it possible to be open to changes to the solution while focusing on the desired business outcomes. In fact, Agile encourages testing all our assumptions to maximise early learning and to minimise risk. Apart from early feedback risk is managed through ensuring that core functionality is built first, a concept that is not necessarily required in iterative development.

 

Agile fosters communication and collaboration

Iterative development retains a phased approach (analysis, design, build, test) and promotes the existence of functional silos. Functional silos communicate through written handovers, often “thrown over the wall”, which are a known breeding ground for misunderstandings and omissions. At best BAs, designers, developers and testers will co-ordinate their work with each other but they will never truly collaborate. 
 
Agile relies on self-organising, cross-functional teams responsible for and authorised to reach a shared goal. Not only will this reduce the number of handovers and miscommunication it will foster real collaboration where the whole is greater than just the sum of its parts.
 
Overall Iterative development doesn’t force organisations to change and while some might say that’s a good thing I think Agile has so much to offer that it’s well worth the effort and learning curve. 
 
 

Sandy Mamoli
8 Comments
  • Richard Barrett
    Reply

    As someone who is process agnostic, utilising both Agile and Iterative methods,Hybrid or whatever process is appropriate for the situation and has implemented delivery platforms for many years, I unfortunately I have to disagree with some of your conclusions which I feel are overly simplistic and too generalised. I will not compare all Agile and Iterative approaches but will use two of the most common approaches Scrum (Agile), which is the most recognisable when it comes to Agile, and RUP (Iterative) to illustrate my points.

    To say that Iterative is just a mini Waterfall process which retains the A/D/B/T approach is a common misconception when the process is not fully understood. To put this into context in simple terms, within RUP, A/D/B/T tasks are performed throughout the SDLC, it is just the intensity of a particular task which differs depending on how far you have progressed through the SDLC.

    RUP is incremental. Its core principles are built on the premise that knowledge and understanding of what you are developing increases throughout the SDLC and change accommodated while still working towards an end goal. It’s very easy for the end goal to keep getting further and further away in an uncontrolled manner using Scrum.

    Also fundamental to RUP is delivering something which works sooner rather than later and to gain feedback early during each iteration. You can adjust the number of iterations to suit the project and therefore the amount of feedback you can receive. But typically this approach requires a little less participation from a usually very busy client than with Scrum, and is therefore a good compromise.

    This next point ‘building the right thing’ is probably where I disagree with your conclusion the most. Core to a process such as RUP is prioritising your requirements, building core functionality and addressing risk early. You can even forget about the process you are using, this is project management 101, it’s a best practice which should be performed no matter what process you are using, and not the preserve of Agile processes. This is also where Agile such as Scrum can come unstuck because its a light framework and doesn’t require much upfront definition. This can make it difficult to understand the true requirements, dependancies and risks for a project. Imagine this scenario, the architecture of the system you are developing is an iceberg, the user stories (features) will form the architecture of the iceberg which protrudes above the waterline, yet 90% is still under the water. Scrum doesn’t help you with this 90% by not defining the core architecture early so features can easily built upon it, potentially leading to a lot of refactoring and throwing away of code.

    Using Agile such as Scrum doesn’t necessarily lead to better quality and neither do you have to wait to until the end of an iteration to begin testing. As already mentioned above, with RUP you continually perform A/D/B/T throughout the SDLC. Also an iteration is no different to a sprint, it has a start and an end and you have a certain amount of features to deliver within that period, and there is no construct within Scrum to guarantee that you won’t only complete 90% of the required deliverable during that period. Because of the nature of an Agile process such as Scrum, I.e. you have given little consideration to the architecture upfront, plus you are refactoring and throwing away a lot of code, so it is very easy for bugs to be introduced/reintroduced. So, you need to spend a great deal of effort on testing, and automate testing as much as possible because the amount of testing required can quickly become unfeasible to undertake manually. Now, the testing that is to be automated will only be as good as the tests that have been written, a potential problem in its self, so its always good practice to fully test your system at the end even if you have been testing throughout the SDLC. If your testing strategy has been implemented well throughout the project, then you should discover few issues at the end and be quickly completed. A common implementation of Scrum in particular with product development teams is a 2 week build sprint followed by a 2 week test/fix sprint and if staggered across 2 teams can result in a release every 2 weeks.

    RUP supports feedback and learnings and provides you with the framework to handle it. It’s core to objectives, just accommodating change, risk, delivering sooner rather than later etc. Scrum on the other hand is very lightweight so one person using Scrum handle things in slightly different way to someone else, and by its very nature you are likely to have more learnings to implement because it doesn’t give you much of a framework to work with. Also, the length of an iteration can be as long as you want it to be, its up to the person running the project, there is no requirement for it to be a certain length.

    Not building your core functionality first comes back to project management 101, if you need to be told to do it, then you are in the wrong profession.

    Cross-functional teams are not the preserve of Agile processes and their use has been around for many more years. Also, there are many more organisational structures than just functional and it is these structures that typically do more to influence good communication and collaboration than a process implemented on a single project.

    Having implemented Iterative and Agile processes in many organisations whom have a traditional Waterfall approach, I can tell you, implementing an Iterative approach does force change but will receive less resistance and quicker adoption than going straight to Agile, and allows for a more gradual adoption strategy.

    What I would like to make clear is that I’m not saying that one of these approaches is better than another, they both have their merits and drawbacks and each are better suited to different scenarios. It is also worth noting that they do work well together, I.e. if you use RUP as your primary process and Scrum in a supporting role and overlaid on top mapping sprints to iterations. In this scenario RUP gives you a solid framework for how to do things, while Scrum helps you keep the team focused, maintain a consistent work rate and address issues quickly with daily stand-ups.

    And lets not forget that Agile hasn’t invented anything new here, it’s just a term coined for certain processes that already existed and had been in use for many years for beforehand.

    Lastly I’m sure your client has a good grasp of what can be implemented ‘successfully’ within their organisation, and is not weaselling-out. If you used the points outlined in this article to justify the change, I would understand their scepticism.

    April 13, 2013 at 10:54 am
    • tommyboy14
      Reply

      Good post!

      And yeah, implementing a traditionally Iterative approach does pose change. However, implementing an Agile approach not only poses change, but it can completely derail an MIS org.

      The fact is, with the advent of offshoring, with the advancement of business analysis, and with the maturity of Project Management, Agile/SCRUM is typically a square peg, and evangelists try at all costs to fit it into the round hole. That’s what’s happening at my org now.

      May 31, 2013 at 12:07 pm
      • Rdlake
        Reply

        Man! What kind of acid are you on? Someone this hay wired is bound to wind up in the news with bad results. I feel sorry for you

        May 31, 2013 at 3:30 pm
  • tommyboy14
    Reply

    Hate to break it to folks, but Agile/SCRUM, like just about every other method or model, is based on Waterfall. SCRUM sugar-coats the process with proprietary jargon and process, but make no mistake it’s all waterfall. The only thing that changes (whether waterfall traditional, iterative, or methods like SCRUM) is the SIZE of the waterfall.

    In every instance, whether you officially track it in a schedule or not, you are doing the same exact things: Analyzing, Designing, Building, Testing, Deploying, and Stabilizing. With SCRUM, you are just stressing your teams to do them MANY MANY times with smaller scope, within unrealistic timelines.

    Take a team that is flawed at basic waterfall, in any of the phases outlined above… Now put that team in an iterative project, or an Agile project.

    They will fail for the same reasons.

    It’s all the same… Except that in typical organizations, SCRUM is unsustainable long term for a team, ESPECIALLY if you’re offshoring.

    May 31, 2013 at 12:00 pm
  • tommyboy14
    Reply

    Let’s also keep in mind context…

    Agile was created by a bunch of really smart American programmer/engineers, and the start of the offshore model being viable…

    Now match up those facts with the tenets of Agile and you come to some real interesting realizations about that method.

    May 31, 2013 at 12:02 pm
    • Rakhsup
      Reply

      Excellent comments/points made from real experience…i wish technical delivery managers across the globe develop this kind of wisdom some day very soon….

      September 20, 2015 at 7:00 am
  • John
    Reply

    Great post. It is the best explanation of the differences between the iterative method and Agile I have read. Your writing is excellent. Please write more!

    April 21, 2015 at 6:48 am
  • Pingback: Agile: Iterating Your Way to Shoddy Work? | Fusion Alliance
    Reply

Post a Comment