Agile - The New World Order - Event Summary
St Leonards Hall, Edinburgh University, Edinburgh, 14th June 2011
Agile project management practices are becoming a recognised standard in the software industry. Is this because agile techniques are so much better than the more traditional project management approaches or is it because the old project management methodologies don’t work? The reality is probably a combination of both – and in these times of heightened awareness of the requirement to ‘do more with less’ or indeed ‘to do more differently’, agile is enjoying a surge of interest.
Agile’s iterative steps where chunks of work form stepping stones, allow for tangible hands-on ownership by the business as the software is built. This visibility to the business ensures deliverables are on the right track, thus enabling better control of actual and future investment. And whilst businesses across all service sectors – public and private – remain under pressure to do things differently agile looks set to become an increasingly important part of the service delivery model.
As evidenced at the NCCs Agile session in Edinburgh – a smaller but very engaged group with only a very small number of the 25 delegates currently using and actively working on Agile projects – there was nevertheless a good degree of awareness of agile methods and a positive feel towards the agile approach in the room. All of the delegates seemingly sharing a drive to explore how to do things differently – interesting and unique to this session none of the assembled had tried but failed to realise the full benefits of an agile approach. This made for an interesting exchange of thoughts and experiences on current business priorities and operational challenges.
The status quo
Before we explore the agile world, let’s take a moment to explore where the traditional management techniques all too often fall short.
Firstly, what do we really mean by ‘traditional project management’? This is a development methodology that treats software development as if it were a mechanical process of turning requirements into specifications, into lines of code; and as if every detail of this process could be planned in advance and measured to arbitrary precision. Add to that rigorous documentation, staged processes and fixed deadlines. Worst of all, the interaction with the business in the traditional approach is very inefficient. Often limited to a project manager’s report that states, “74 % of the features are implemented and ready for testing, the next 10 % will be implemented by 1st October.”
This may be comfortable to the business, because it feels that most of the project is done. However, the failure here is to realise that the most important and most complex features could be in the not yet done 26 % of the features. The business also assumes that there are no defects in the three quarters of the work that has been completed.
Suppose now that the deadline falls when the project is, say, 90 % complete. The business assumes that the 90 % includes the most important features and that the features match its expectations. What all too frequently follows is bitter disappointment.
And here’s what the research tell us
We all have anecdotal stories from development projects, and indeed we all have our own experiences to draw on: despite well crafted project plans and good project managers, the end result is way off what was expected when the expenditure was sanctioned. The statistics are alarming, according to research from a variety of established sources (Gartner, Forrester, Accenture):
- 60 % – 80 % of project failures can be attributed directly to poor requirements gathering,
- 40 % of system problems are found by end users,
- up to 80 % of budgets are consumed fixing self-inflicted problems,
- just 16 % of development projects close within targets of cost, time and quality,
- Cost overruns of anywhere from 100 % to 200 % are common in software projects.
So is the industry learning from these costly mistakes? The Standish Group alerts us to a step backwards after some years of gradual improvement. Standish has surveyed over 50,000 completed IT projects since 1994 and is acknowledged as the most referenced source of project success – where ‘success’ is defined as projects delivered on cost budget, on time budget, and with expected functionality. The headlines make stark reading:
According to NCC’s recent survey, FTSE-250 companies spend an average of 6.4 % of their turnover on ICT, with 43 % of this spent on hardware, software and services. This means that on average 2.75 % of turnover is spent on hardware, software, and services. For the purposes of this discussion, let’s assume 40 % of this spend is allocated to development.
If we take the Standish findings that 24 % of development projects fail, applying this to a £500m turnover business, the cost of failure is £1.3m. Recalculate this as a percentage of gross margin and it starts to become painful and something shareholders would take an interest in.
And remember, this number only represents a small part of costs incurred by an organisation when a project fails – we can’t ignore opportunity costs – the costs of unrealised business benefits and the fact that other projects didn’t proceed due to capital rationing. They are not as easy to arrive at as direct project costs, but maybe it's time that this real cost to the business became part of the analysis.
The new world order…
Agile promotes a radical change to this process; a change that allows the business to control the development process, the technical team to maintain excellent quality of the code and for both sides to discover defects as soon as possible. With an agile approach, the business interacts with the technical team regularly and would be able to drive which features it wants to have implemented first. Consequently, if the project is incomplete by the deadline, at least the business would know exactly what features it contained. Furthermore, as a result of the short iterations in an agile approach, the business will have seen and tested the implemented features; this reduces risk and, in a sense, ensures that the business gets its money’s worth of software services.
Let’s be agile
Agile project management methodologies set out to minimise waste. By ‘waste’ we mean unnecessary work. Agile makes no distinction between the types of work: requirements, specifications, code; it is all the same to agile. This is certainly appealing: as the business, you get exactly what you pay for; as developers, you do not spend time and effort writing code that you do not want or the business doesn’t need.
Based on the principles of ‘lean thinking’, agile provides an intelligent and customer value orientation to software development, transforming culture and expectations of developers and users alike to achieve ‘more with less’.
The ethos is simple: the priority is to satisfy the customer through early and continuous delivery of working software, working collaboratively with the customer, thereby ensuring features and functions required are delivered and we don’t over engineer or overspend a budget on unwanted features.
From a cost perspective, the ‘traditional’ and ‘agile’ approaches to cost-v-value maybe considered as follows: delivering the software solution and features the business needs today is the priority. Most studies concluded that 25 % of the features are ‘must haves’ and 75 % were ‘nice to haves’. Other research states that 25 % of the features are changed in a year because of the dynamics in the business.
There is also evidence that up to 60 % of features are never used (over-production). Excess or over-engineered features in software are a consequence of traditional waterfall methods. Product managers have one shot to get all the requirements out at the beginning of the project, so we force them to think of absolutely everything. As a result we spend large budgets on nice to have features or features not used. Agile, focuses on delivering the most important features every ‘sprint’ – a sprint takes the form of a fixed period of work, a maximum of 30 days, to deliver and reengage with the business. In a sprint, a team develops deliverable software that can be used by the business. In the next sprint, the next important features can be built, or depending on the experiences in production, the business can decide to build other new features or stop developing others that were previously planned.
Agile provides a set of tools to prevent the over or under production of features. The focus on developing top business value and deploying your first implementation after a few sprints in production has positive impacts: short term financial control, clarity, focus and insight into current performance, the chance for realignment and full visibility on the collaboration between customer and supplier.
Comparing these approaches and the research findings in budget overruns, the waterfall approach risk is higher because the project horizon is typically in months; in the agile approach the chance of budget overrun is significantly smaller because the project is broken up in small projects – or sprints – giving clarity and visibility.
‘Agile Financial Times’
In addressing the legacy of unsuccessful development projects and cost overruns, agile represents a culture change, and while it usually originates in the IT function, it is a cultural change in the entire organisation as to how projects are developed and implemented into the business, that is required. The aim is to have ‘agile organisations’, not ‘agile IT functions’ - the engineering rigour and expertise is necessary, but not sufficient condition of agile.
In an agile organisation, the IT team and the business work together to build software. Without the involvement of the business – and an understanding from the business that software requires continuous improvement, it is simply impossible – the results will ultimately be disappointing. For those who say that ‘we need to analyse the details before we can start coding, without complete analysis, we can't be sure that we'd not miss anything’, the danger is that they will suffer from ‘analysis paralysis’ and lose time. The solution is to analyse just enough to get started; then demonstrate to the business. The customer knows what they want.
The integrative model of project delivery that agile represents has risk sharing features, borne out of collaboration as one team to develop, define and deliver the project. The focus is on collectively achieving shared goals rather than meeting individual objectives. Success is measured by the degree to which common goals are achieved in line with the project budget. In times of austerity, or indeed in times of plenty, agile represents a business focussed way forward for software development.
The choice is now yours; you can explore agile, and in so doing give the business what it wants and needs, give the teams the drive to be excellent, attract the best talent. Or, you can keep toiling with other project management approaches.
And finally, the full slide deck for the session can be accessed at: www.ncc.co.uk/agile-2011-06-14
Additionally, here’s some very useful links;
- Agile Project Management Guidelines
- Java vs .NET – Technical Guidelines
- A number of agile case studies & white papers, including the Agile Financial Times www.cakesolutions.net/technical_library.html
- Agile Community www.agilefinancialtimes.co.uk
- Agile World Conference - Leeds www.cakesolutions.net/conferences.html