The Values of Agile Vs Waterfall, how to get the best from both.

As a Project Manager you may (or may have not) been exposed to various methodologies for running projects throughout your career. Methodologies are a big deal in Project Management Land. But why are methodologies so important to us and why do we hinge everything around them? Do they even matter that much? I’m currently in a business where if I’m frank, the true understanding of each methodology isn’t really understood by many and I wanted to outline it here.

Methodologies can be a bit like faddy diets, there’s always a new one just around the corner which is meant to be the saviour and you should forget everything else you are told, and use this not that etc.!  These conversations are usually lead by a methodology evangelist, it’s a bit like the fight between Apple & Samsung, or Coke & Pepsi. Everyone has a strong opinion to what’s right and what’s not, what you should do and what you should not!

There’s two very common ones you’ve almost for sure heard of though. ‘Waterfall’ – the archetypal and common methodology which suits many, its staggered, easy to plan for, structured, and done in manageable chunks. Then there’s ‘Agile’ – a much more iterative and evolved but complex methodology which many companies fail to truly understand or are geared up to accept. We’ll talk about both in this post and the benefits of each.  We’re going to discuss these topics then;

• What are these Methodologies
• Which methodology should I use
• I want to use Agile, but it’s tough to implement
• Does a methodology guarantee project success.

What are these Methodologies – Waterfall

Waterfall is very much a phased approach. You do a bit of work, a bit of work that’s been defined and planned. You don’t do the ‘next bit’ until the previous bit is done, you all agree and you move to the next section. That’s basically the essence of Waterfall.

People like this approach because you can see what’s coming on the horizon & therefore plan for it. Businesses like clarity and visibility. You can easily spot problems up ahead and you can scale resource as needed. Plus, you have governance and security baked in as you don’t proceed until you’re all happy with the last ‘bit of work’.

So, this is a safe methodology. If followed correctly it’s low risk by comparison.  You can pull out at any point and businesses often favour this way of working because of that. The problem is it’s slow. It doesn’t take advantage of running multiple tasks at the same time and it avoids being efficient because it dictates the pace of progress in a very linear way more often than not, failing to truly integrate with the nature of digital. It also demands a fair old amount of documentation (sometimes for documentations sake) and you can easily get wrapped up in the governance before you’ve actually done anything.

And you see, that’s always the problem with Waterfall. A fantastic methodology if you’re building a bridge, putting a car together (you can’t put the wheels on until you’ve installed the axles etc.) but when you’re developing something that evolves like a website, this methodology doesn’t always work.

To truly understand why Waterfall may not be the right way for technology or digital projects you first have to understand the nature and essence of a digital project. We’ll for this example, consider the build of a website;

Did you know a website build contains several factors? A front-end design or the ‘look & feel’, (the way the site will look), the back-end functionality (the way it will work and the functionality it will provide) the UI design (the User Interface), how you interact with the website, the UX (User Experience) how it will ‘feel’ to the user.

It’s a bit like an onion, with many layers. All different disciplines (layers) usually done by different teams of people. Unlike building a bridge, where you can’t start building until you’ve laid the foundation and you can’t lay the foundation until you’ve got planning permission, a website can be started at any point and in any order!

You can speak to the users by way of a focus group for example, and start to cultivate ideas & desires of how the website should look and work. This would then inform the design, the UX and UI and so on.

Or, you could build a basic website, and then seek feedback before layering in the design, look & feel. Building an e-Commerce website? Maybe you should start with your product catalogue etc.! It’s very much like a painting. You start with the base layer, and work in the detail over time, it just doesn’t always matter which order you do it in. You can start straight away with huge amounts of visual detail and work backwards, you can start in the middle and work out either side, or you can focus on key areas such as functionality or your product catalogue, and build the website around that – the ways of building a website or piece of software are endless.

You can hopefully start to see then that a website is a complex beast, and there’s no specific start, middle or end point. And that’s where ‘Agile’ comes in. ‘Agile’s basic principle is you build something quick, get it out to users, get their feedback and then iterate. Rinse, repeat and you improve with a constant feedback loop.

It’s never perfect, it comes with bugs, but you finesse, refine & improve as-you-go. This is an efficient way and avoids building something (code is expensive to write) to find out later down the line it’s wrong. This is called technical debt. This can plight a lot of technical digital projects. As failure to plan contextually as you go, instead opting for all the planning up front but with no context, means you end up with a product that may not be what people want or need.

Huge digital projects often fail (think NHS or Council websites). Whilst they are complex beasts a common issue I’ve seen deep in the trenches is because there’s this ‘grand reveal’ at the end and the customer gets something they didn’t expect or could foresee when they wrote the cheque. Failure to bring the customer along on the journey is such as short sighted view, and whilst it’s understood this is to protect the ‘design by committee’ risk, I still think its a failure.

So, by revealing early, building in small chunks and showing your customer how it’s going on-the-fly as you go, you can all work together to get something you all wanted, expected, and what meets the desired need. And if somethings going wrong, you can spot it early enough before it becomes a major problem. These are the distinct beneficial tenets of Agile over Waterfall.

Sounds perfect, right?

Well, the thing with Agile is culturally it requires a business to have faith in human nature, to allow the faults technological complexity brings with it, and to put your name on something which starts off less than perfect. The best analogy I can think of is your iPhone you currently have in your pocket (unless you’re an Android user – we don’t talk about you! ) each month, you receive an update which fixes or patches the operating system fixing a myriad of bugs, problems and security issues. The product you bought in the shop was deemed good enough to sell to you, but the developers at Apple knew that an operating system is a constantly evolving beast with many changing moving parts, so you accept this fact and continue to apply updates month on month.

It’s the same with your website. If you strive for perfection up front, you’ll rarely get to a finish point. So, agile forces you to be less than perfect. You take risks by minimizing documentation advising on how something will look, feel or work in favour of actually building a real physical example that people can smell, touch and play with. Sure, it’s not perfect, sure people will have questions and lack clarity to start with, but over time by including the customer in the design of the product, you ultimately end up with something that suits all.

Agile dictates the focus on building quickly, learning often and iterating for improvement. This cycle is often fortnightly or monthly to match a business’s output, and it also demands that activities and tasks which can be started simultaneously are, for the benefit of expedition.

In the example of a website, using Agile, you’d probably work something like this;

• Start building straight away based on a very loose spec from the client/customer
• Artists, designers and visual creatives will start to create mock ups in draft, sketching ideas and putting those ideas to the customer to allow them to feedback and drive further designs
• At the same time, the point above may be instead written directly in code, giving physical working examples of the end result. Again, seeking feedback from the customer.
• Testers would test as code is made available, with understanding that bugs may be present
• Documentation will be loose, minimal only and not drive the project.
• Failure or less-than-perfect is accepted, with two week sprints managed and not looking beyond that. Get to something that works (barely) and improve over time.

The above could sound like chaos, but it’s actually very efficient. Workstreams can be progressed and getting customers on board right from the off ensures the true needs of the customer are taken into account. This builds a solid relationship of trust.

The business would need to have faith & trust in its developers, its test teams and everyone involved that the right thing is being done at the right time. Always centered around a clear answerable objective – ‘what are we trying to do?’

If you can as a business clarify that into a couple of sentences, in a language your mum can understand – then agile can work. Having everyone know the answer to this question helps keep everyone honest & ensures decisions are made with the right intention.

The bad things about agile for when you’re a large slow moving complex organisation can be;

• Your people don’t trust this way of working as it seems too loose. Not having long term visibility can cause panic through perceived lack of clarity.
• Agile requires everyone to roll their sleeves up, fully adopt to the methodology and accept less-than-perfect
• Agile requires faith. If you’re people are not bought into it culturally or from a process perspective, it doesn’t work. You need infrastructure, seating arrangements, easy honest communication and a fairly flat hierarchy (amongst other things) for it to work properly. Politics kills agile, every time.

Now let’s compare the above to waterfall.

If building a website with Waterfall You would;

• Start with a technical & functional specification. You’d agree these documents with the customer & then get it signed off. You don’t start building or designing anything until everyone’s in agreement.
• You design the front-end look and feel and how things will behave. You get sign off from everyone before you continue.
• You pass the designs to the coders who see it all for the first time, having previously not been involved. They build as they are instructed as per the technical spec and the visual guidelines.
• A beta period is reached, and then testing starts.
• You bug test, fix, and then finally show to the client.

That sounds sensible, right? Waterfall comes with many problems too though so hold those horses. These issues Include;

• New features or things only-just-thought-about cannot be included
• Documentation takes time. Sign off takes time. This can create huge delays in project timelines and customers change their mind & intention/desire when they get bored and see little progress.
• The whole pass-the-baton thing from team to team creates great surprises. The developers don’t have visibility into what’s been designed, the designers don’t have a clear idea of what’s technically possible and somewhere in the middle are expected to meet.

There’s many more issues and we could talk about this all day, but you get the gyst. I typically follow the Iron triangle method to bind my project together. You can use this principle inline with a methodology. It follows the rules as below;

So what do business’s do right & wrong?

So, what a lot of businesses tend to do is they merge both. A bit like splicing an ear onto a sheep. It doesn’t look good and doesn’t work that well either. Both agile & waterfall are always at odds with each other, and subsequently the project falls into disarray. You can attribute this situation to a lot of failed IT projects. The ‘merging of two different gearboxes’ if you will.

So what is recommended then?

Well for a start, as a Project Manager, learning the nuances & pro’s & cons for each of these two methodologies is a must. Then being able to spot when a business is trying to do both and failing miserably is a valued skill. There’s also a fine knack between running agile but with waterfall aspects to help appease a business – this is a risky game though. Referred to as a ‘blended approach’ it rarely is and if you find yourself in the middle of a bad ‘blend’ then you better run for the hills. See my top ways to make agile & waterfall play a bit more nicely together;

Sprint management – Zealous account management

Explicitly manage the expectations in writing at the start & end of every sprint. Further, have a customer sign off for each featured delivered during the sprint. Waterfall safety, but in an Agile style.

Worst mistake to make: Head in the sand behaviour hoping that the issue will take care of itself.

Change order Bonanza!

When changes are made, get them documented and ensure the vagueness get’s squashed & ironed out.  Change orders should include both schedule and cost impacts, as well as feature details.

Worst mistake to make: Delay or even skip the issuance of change orders, leaving it to a ‘verbal understanding’ that will quickly be forgotten.

Explicit closure of discovery

The project requirements document produced by your BA or discovery team should include verbiage indicating that the contents are the totality of binding requirements for the contract and include SLT signatures and workstream owners to sign and date.

Worst mistake to make: let it slide, accepting new requirements at any stage of the project without a change order.

In Summary, you can in fact use both methodologies, but you really need a clear direction & vision in terms of brief, or spec, or what you’re trying to achieve. If it’s open ended, Agile can really work, but it needs the foundations in place to be successful. If you’re a waterfall organisation, that’s fine, don’t sweat it. However, embrace diligence fully & do it properly. Be that Project Management, PMO, Governance or anything else which comes with Waterfall. Mostly though as a PM, get to know both.

Daily Stand ups, using Kan Ban (another methodology-ish, we won’t talk about today) using Agile tools can all help a waterfall organisation, you just have to know what tool and process to use and when.