The Importance of End to End Thinking – Even When Agile

By Steve Messenger | 13 July 2015

Debate is currently raging over the Government’s implementation of Universal Credits. Delays and changes do not constitute failure, representatives claim, but rather evidence the adoption of agile project management to realise the earliest possible benefits. Steve Messenger, Chairman of the DSDM Consortium, reflects on what it really means to be successfully agile in today’s environment.

I have recently been using the analogy of an agile sports car with an agile engine to explain why, when embarking on an agile journey, it’s not just about the car. It’s also important to have the right drivers, to know where you are going and why you are going there, to have waypoints along the route to check progress and to ensure good road infrastructure to drive on.

There is currently a lot in the press about Universal Credit’s agile journey.  Amending my analogy a little, I would liken this to a group of enthusiasts undertaking a circumnavigation of the world without ever having previously stepped into the water. It’s a brave, perilous journey to undertake and it took courage to consider it. Whilst there are problems along the way, some of the ships are now sailing into port - albeit a little battle worn.

Universal Credit is of course a large programme of change, to use the vernacular of Managing Successful Programmes (MSP). It not only delivers a new computer system but also undertakes transformational change to the way in which the whole benefits system operates. So no doubt MSP best practice has been used to define the benefits, the blueprint and the tranches to realise those benefits. In fact, adopting this approach should already be somewhat agile as we are decomposing the problem into more manageable chunks and focusing on the benefits to the community - both claimants and local and central government. If we make the tranches small enough, the benefits start to accrue earlier.  I wonder though whether this has been the case, since MSP can be construed as requiring all design up front, although in reality this is not true.

For instance, an agile approach to MSP would define our direction and our major waypoints and let us understand how tranches can be prioritised to realise benefits often and as early as possible. There are proven prioritisation techniques such as MoSCoW that can help with this, taking into account the bad weather and unfriendly communities (resistors to change) that may impede progress. It would also suggest making some early forays to get an idea of potential progress and problems, then using this information to re-evaluate the plan. We can then be strong and not commit to unachievable deadlines. Knowing our direction and major waypoints gives us leeway to let the plan evolve as we find out more, taking on board cargo we forgot or is now vital and jettisoning that which no longer will add value.

It follows that before undertaking the journey we should consider the following:

  • Understanding enough about the journey to be able to determine what has to be done to be successful and how long that will take.
  • Having good crews experienced in handling similar conditions.
  • Having a good Admiral and Captains who can direct the crews to the endpoint without interfering too much in how they get there, but who are also ready to deal with those unfriendly communities on the way.


The Journey

The agile engine that moves the ship forward is sound and will go at speed, but good progress also relies on a well-planned route. Otherwise how can we know where to expect bad weather, pirates, or hostile communities?  How do we even know if the journey can be completed in the time allotted?

So what can we do to make the journey smoother?

The People:

Understand the community of stakeholders - are they accessible and also amenable to change? From a business perspective, who makes the decisions and who really does the job? Who determines success?  Actively involve key stakeholder representatives. I once worked on a Housing Benefit system that failed not because the system was wrong, but because we removed the part of the job that required skill from those using it and therefore significantly changed their working life. 

The Environment:

Understand the legacy and infrastructure and what we will demand of it. Make sure stakeholders from these areas are also consulted and involved. There is no point in building a system in record time if there's no data to go in it, no servers to run it on and inadequate power to run it efficiently.

The Individual Ships:

Identify the projects that will provide the wherewithal to fulfil the benefits. Some will be naturally agile (iterative and incremental) - for instance those parts with high user interaction. Some may lend themselves to more traditional approaches – such as building the required infrastructure. Some may be a blend. Migrating data is an obvious example. It is highly unlikely that every part will take the same approach.

Just as each individual ship will plan its journey around making adequate preparations to be seaworthy, each of the projects should do the same. If we understand enough to know roughly how much we can achieve in the time allotted, this will actually tell us whether or not we can be successful. It is better to stay in port renegotiating time and resources than to set off unprepared and run out of food and supplies mid- Atlantic. Since we are fixing our time and resource, we need to understand the limitations of our journey and consider our options. There will be lands along the way that we have to visit, but others may not be so important. Giving ourselves leeway lets us plan contingency without the need for more money, time or resource. Of course, the contracts we put in place have to reflect this!

The Crews:

Whilst it may seem efficient to have our ships manned by third parties, keeping them aligned and on course can be a difficult job and requires the right structures to be in place.  The crews should be the experts in their fields, but they are there to serve the business area experts who sit within the organisation. To be most effective, teams should include both full time sailors who understand how things work within the organisation and are committed to the mission, as well as third parties that bring specialist skills or ensure that the ship is fully manned. They are all ‘in it together’ and succeed or fail based on their joint capabilities and joint commitment. We must ensure that the contracts in place with the third parties reflect this attitude - the importance is on reaching the end goal, not necessarily every port of call on the way. 

The Admiral and Captains:

In my opinion the leadership - the Admiral and Captains - are always better drawn from within the organisation if possible, since they fully understand the purpose of the mission and eventually will have to live with the consequence, whether this is success or failure.  The most important thing is that they are good leaders, not necessarily that they have experienced agile before.  The use of coaches can help them to move in the right direction, and good leaders always learn quickly.

As long as the direction is clear, each crew can steer and run its ship more efficiently and effectively than anyone else. However, they do need to know when there is danger - crossing another crew's path or even steering in an opposing direction to the rest of the flotilla. This is where good communication channels between the ships are important, particularly when some are delivering supplies to others.

In Conclusion

So should we define every detail of what the crew needs to do?  No – this is not the agile way. Providing fully detailed specifications up front is rather like giving someone a map of the world when no one has, as yet, circumnavigated it. There are areas that are at best vague, probably unknown, and along the journey new volcanic islands will erupt from the sea and the whole landscape will change. Our ships have to embrace that change. They expect and need to react to information about what is happening around them, but they also need to keep focused on the true purpose and vision of the mission. 

In the development of the System Error report, the Institute for Government jointly funded the Amberhill project as an exemplar of agile in government. Since then Amberhill has continued to be successful and is now entering the third phase of delivery. A critical part of its success is that although part of a large programme, each part has been delivered independently and incrementally - thereby minimising risk and ensuring early return on investment.

Project Amberhill was initiated within the Metropolitan Police Service (MPS) to provide the means of sharing recovered false identity data that resulted from a range of policing activity against criminals engaged in the creation and use of false identities. It shares recovered data with key public and private sector stakeholders to terminate on-going and prevent future crimes.  Amberhill has already delivered benefits in excess of £5million, and further increments of the system are anticipating £1bn savings over the next five years.

So please don't scupper the fast, efficient agile ships.  Rather put in the organisation and governance around them that will ensure they sail successfully.

Steve Messenger is the current Chairman of the DSDM Consortium and has been involved in Agile since its inception, DSDM being one of the signatories to the Agile Manifesto. As well as managing many Agile projects, Steve has implemented DSDM into the highly regulated pharmaceutical industry, particularly during his role as Software Engineering Manager for Mundipharma IT Services.

Key Steps to Becoming Agile

  • Support from Senior Management

Projects and programmes are usually aimed at effecting business change and embrace a much wider context than simple product (often software) delivery. When adopting an Agile approach it is very important than senior management understand and support the chosen approach.  These are the ‘Admiral’ and ‘Captains’ in the article – the people who fully understand the objectives. Without this senior level buy-in there is a high risk that the benefits of adopting an Agile approach will at worst not be achieved at all and at best severely reduced.

  • Firm Foundations

In order to deliver real business benefit early, all Agile approaches advocate incremental delivery.  However, it is essential that firm and enduring foundations for the project are first established that include the perspectives of business, solution and management which, when combined, can provide a clear focus that is both robust and flexible. 

  • Good Communications

Communicate continuously and clearly with all stakeholders.  It has been well documented that poor communication is often the cause of project failure.  There are a number of Agile techniques, such as daily stand-up sessions, that are effective in ensuring that all team members are valued and have the opportunity to communicate and share information.