|From Performance to Value: Measuring in Agile|
From Performance to Value: Measuring in Agile
In This White Paper
June 2015 | Agile Research Network | Agile Transformation
Faster delivery of business value is often cited as a reason for adopting Agile. However, measuring the value achieved through IT development is challenging.
Created by the Agile Research Network
From Performance to Value: Measuring in Agile
Faster delivery of business value is often cited as a reason for adopting Agile. However, measuring the value achieved through IT development is challenging. For those relatively new to Agile, measurement is not straightforward, as traditional project and portfolio metrics are often hardwired into management report requirements and are hard to change. During Agile adoption it makes sense to start by measuring team performance, as this can aid learning. From small beginnings, an Agile measurement approach can iteratively be developed to meet the requirements of the business.
This white paper from the Agile Research Network (ARN) presents the case of a department within a multinational organisation in the insurance sector that adopted Dynamic Systems Development Method (DSDM) in the first quarter of 2013. One of the issues they faced was how to shift from a traditional approach to measuring IT development to one that is compatible with Agile development. The ARN present the challenges they were asked to investigate along with suggestions from the published literature about how to overcome them, and a summary of proposed mitigation strategies. The primary challenge was ‘understanding and measuring value in Agile projects’.
In order to do this, the ARN were asked to investigate the three levels of performance measurement that the department already used, and explore how they may be adapted to be more suitable for Agile working. The three levels we looked at were:
The department already had a well-established measurement process at all three levels for their waterfall projects. As they started adopting Agile processes they wanted to shift their measurement practices in order to ensure that they were capturing the right management information as well as driving the right behaviour. Three types of recommendation for how to approach these challenge areas are identified from the literature:
The measurement of IT products and development processes is a well-established practice, and is a core part of the traditional project manager’s job. As software development is a complex process, some sort of measurement is necessary in order to understand what is going on and to improve. The agile approach to measurement is to focus on improvement and to ensure that value is achieved. Metrics can be devised to assess different aspects of an agile project, for example aspects of team learning, process improvement, product quality, and stakeholder satisfaction. The main point is that in agile, measurement is a tool and not an end in itself.
In this case study we (the Agile Research Network) worked with a department that was undergoing a transformation from waterfall to agile IT development. They wanted to know how best to measure the success of that transformation to ensure that the new approaches they were adopting were delivering value to the company.
We also found that there were differences between how the directors and assistant directors viewed the future of the organisation compared to how middle management, as represented in the strategic board, expressed their preferences for the future culture of the organisation.
Value in the context of Agile IT projects often refers to the delivery of ‘business value’, which includes any benefits that maintain the general health of a company. It is closely linked to economic value, and ultimately may be assessed by measures such as return on investment (ROI). In order to justify an IT investment, a calculation may be required to show that the income generated or the cost savings made will be greater than the cost of producing the software. However, economic value is also embedded in many features of a business. Information technology does not exist in a vacuum; it is a complementary system that works with other factors in the business . Value can accrue, for example, through knowledge production, brand awareness, loyalty, customer satisfaction, and trust. Some IT developments may only yield economic benefits over time and may not show immediate gains. For some projects, such as infrastructure projects, it may be very difficult to measure their precise economic value to the business. A long-term perspective may be required for some developments, such as those aimed at gaining competitive advantage, improving supplier relationships or achieving strategic alignment.
In the rest of this paper we report our investigation, which starts by looking at the story of agile adoption in the department. We explore when, how and why they started using agile, their existing performance measures, and how these relate to project and personnel management. We identify the specific challenges they face in the current context, and we make suggestions about performance measurement from the literature. Finally we come back to the issue of value, and we propose an approach they can adopt for identifying and measuring value.