Evaluation of a Scholarly Peer-Reviewed Article DIT-R8921

Research article

Factors affecting the successful realisation

of benefits from systems development

projects: findings from three case studies
Neil F Doherty1, Colin Ashurst2, Joe Peppard3

1The Business School, Loughborough University, Loughborough, UK;
2The Business School, Durham University, Durham, UK;
3Cranfield School of Management, Cranfield, Bedfordshire, UK

Correspondence:
NF Doherty, The Business School, Loughborough University, Loughborough, LE11 3TU, UK.
Tel: þ44 01509 223328;
Fax: þ44 01509 223960;
E-mail: [email protected]

Abstract
The return that organisations derive from investments in information systems and
technology continues to disappoint. While there is a very significant body of literature on
the factors that should facilitate a successful outcome from systems development, there is
growing concern that these prescriptions are not having their desired effect. In this paper,
we argue that the success of a systems development project should be measured in terms
of its ability to deliver meaningful benefits, rather than the timely delivery of a technical
artefact, and therefore organisations should adopt an explicit and proactive benefits
realisation approach when investing in IT. Consequently, we sought to explore those
actionable factors that might facilitate the effective realisation of benefits from systems
development initiatives. Three organisations were identified that claimed to adopt a
proactive approach to benefits realisation, and detailed studies of their systems
development practices were conducted. Our analysis found that whilst one organisation
had been successful in its adoption of a benefits realisation perspective, the other two had
not, and this allowed us to identify those factors that helped to explain this difference in
outcomes. In short, this paper makes an important contribution by identifying how a sub-
set of traditional systems success factors might be enhanced, to give them a more explicit
benefits realisation orientation. Moreover, it presents a coherent set of principles that can
be used for deriving other factors and practices.
Journal of Information Technology (2012) 27, 1–16. doi:10.1057/jit.2011.8
Published online 9 August 2011
Keywords: IT development projects; benefits realisation; organisational change; ISD success factors;
value

Introduction

T
he context for the research reported in this paper is the
continued high failure rate of investments in information
systems/information technology (IS/IT): a considerable

amount of time, money, effort and opportunity can be wasted
upon IT investments that ultimately fail to deliver benefits
(Fortune and Peters, 2005; Peppard and Ward, 2005).
Estimates of the level of failure may vary, but over the past
30 years they have tended to stay uncomfortably high. More
specifically, it has been suggested that in the late 1970s only
20% of the projects ‘achieved something like their intended

benefits’ (Eason, 1988), and that by the late 1980s, it was
estimated that up to 70% of IS projects could be classified as
failures (Hochstrasser and Griffiths, 1991). By the end of the
1990s, Clegg et al. (1997) reported that ‘up to 90% of all IT
projects fail to meet their goals’. In the last decade, Shpilberg
et al. (2007) reported that 74% of IT projects from 1994 to
2002 failed to deliver expected value, and a British Computer
Society (BCS, 2004) study concluded that ‘only around 16 per
cent of IT projects can be considered truly successful’. An even
more recent survey of IT executives found that 24% of IT

Journal of Information Technology (2012) 27, 1 –16
& 2012 JIT Palgrave Macmillan All rights reserved 0268-3962/12

palgrave-journals.com/jit/

projects were still viewed as outright ‘failures’, while a further
44% of projects were considered to be ‘challenged’, as they
were finished late, over budget, or with fewer than the
required features and functions (Levinson, 2009). Against this
backdrop, it is important that more reliable ways of managing
projects to implement IT investments should be established to
help ensure that these can consistently deliver important
organisational benefits, rather than becoming a drain on
corporate resources.

In response to the ongoing problem of systems failure,
academics and practitioners have sought to develop lists of
those critical factors that, if addressed, might help to ensure
that an organisation’s ability to develop and implement
effective new IS might be radically improved. Such lists
routinely include factors such as senior management
commitment, proactive user engagement, etc. (Plant and
Willcocks, 2007; Wang et al., 2008). Unfortunately, despite
the widespread promotion of such prescriptions over the
last decade, there have only been modest improvements in
IT success, and in too many cases the return from IT
investment projects continues to disappoint (Shpilberg
et al., 2007; El Emam and Koru, 2008). As noted by Cobb
(1996), in his now much quoted paradox: ‘we know why
[information systems] projects fail, and we know how to
prevent their failure – so why do they still fail?’ One possible
explanation as to why systems development projects are
still frequently perceived to be failing, despite our
accumulated knowledge of those success factors that should
prevent failure, may well be due to how we define success/
failure and monitor performance. An IT project is still
often judged, by the project team/management, to have
been successful if the commissioned technical artefact is
delivered, on time, on budget and to specification (Ahn and
Skudlark, 1997; Clegg, 2000; Doherty and King, 2001; Eason,
2001; Markus, 2004; Sauer and Davis, 2010). However, from
the perspective of the organisation, a project will only be
perceived as successful if it ultimately delivers benefits that
exceed the cost of achieving them.

One potentially important mechanism for ensuring that an
IT project is focused upon improvements in organisational
performance, rather than simply the delivery of a new piece of
IT, is through the establishment of a formal and explicit
benefits realisation programme. Benefits realisation manage-
ment (BRM) has been defined as ‘the process of organising and
managing, such that the potential benefits arising from the use
of IT are actually realised’ (Ward and Elvin, 1999). Such an
approach is based upon the growing recognition that the
benefits of IT typically come from the organisational change
that accompanies its introduction, rather than stemming
directly from the possession of a technical artefact (Goodhue
et al., 2002; Peppard and Ward, 2005; Hughes and Scott
Morton, 2006; Peppard et al., 2007). Indeed, a number of
previous studies have attempted to promote the role of formal
and explicit ‘benefits realization’ approaches, for improving
the outcomes of IS development projects, through the
proactive management of organisational change (e.g. Farbey
et al., 1992; Ward et al., 1996; Remenyi et al., 1997; Ward and
Elvin, 1999; Peppard et al., 2007). However, to date, there
is little evidence that organisations have been able to translate
these prescriptions into effective working practices (National
Audit Office, 2006; Ashurst et al., 2008). Consequently, there is
a pressing need for more empirical insights into how

organisations might most effectively incorporate an explicit
‘benefits realisation’ perspective into their existing procedures
for the design, implementation and operation of IS. One novel
and potentially promising line of enquiry might be to explore
how our existing understanding of project success factors
might be modified, if such factors were far more explicitly
focused upon the realisation of benefits, rather than the
delivery of a technical artefact. To this end, we sought to
investigate how traditional success factors might be tailored to
explicitly facilitate the successful realisation of benefits from
an IS development project. In so doing, we sought to forge
productive new links between two substantial bodies of
literature – IS success factors and benefits realisation
management – that as yet have failed to deliver on their
promise.

In this paper, we focus on the detailed findings of case
studies from three public sector organisations. One case
study is of particular interest, in that it provides an example
of an organisation that has been successful in the adoption
of a benefits realisation approach, when managing its
investments in IT. This positive case example is contrasted
with two other organisations that are also attempting to
adopt a clear benefits realisation focus, but with far less
success. The structure of the remainder of this paper is as
follows. Firstly, we set out the foundations for the study
in a review of relevant literature. We then outline the
overall research model and the research methods adopted.
Next, we provide brief background on the organisations
explored in the research, following which we outline the
findings related to factors that have facilitated or impaired
the adoption of benefits-driven practices for IT-enabled
change, before finally relating these findings to the extant
literature and then reviewing the implications for practice
and further research.

Review and research objectives
The purpose of this section is to provide a critical overview
of the literatures pertaining to the success factors for
systems development and the realisation of benefits from IT
projects, before highlighting the gaps in these literatures,
and then presenting the study’s objectives.

Success factors for systems development
The primary driver for the research into success factors for
systems development has been the continuing failure
of organisations to realise the full potential from their
investments in IS/IT (Standish, 2001; BCS, 2004). Against
this backdrop, a significant body of research has been
conducted, over the past 30 years, in an attempt to identify
and verify those actionable factors that are critical to the
successful outcome of complex IS development projects.
Most of the early research contributions attempted to
derive generic lists of those factors that would be equally
appropriate for all classes and types of information system
(e.g. Cerullo, 1980; Rademacher, 1989; Yap et al., 1992;
Sauer, 1993; Willcocks and Margetts, 1994; Li, 1997). In
more recent years, the tendency has been to focus studies
more explicitly on success factors for specific categories of
information system. For example, the success factors for
CRM systems (Kim and Pan, 2006; King and Burgess,
2006); ERP systems (Somers and Nelson, 2001; Plant and

The successful realisation of benefits NF Doherty et al
2

Willcocks, 2007; Wang et al., 2008); executive IS (Poon and
Wagner, 2001; Salmeron and Herrero, 2005) and global
systems (Angeles and Nath, 2007; Biehl, 2007) have all been
previously studied. Despite the significant period over
which success factor studies have been published, and the
variation in the technologies studied, there is a surprisingly
high degree of consistency in their findings. In particular,
nearly all studies have highlighted the importance of factors
such as: active user involvement (e.g. Rademacher, 1989;
Yap et al., 1992; Kim and Pan, 2006); senior management
commitment (e.g. Sauer, 1993; Li, 1997; Wang et al., 2008);
appropriate staff training (e.g. Milis and Mercken, 2002;
Biehl, 2007); the expertise and capability of IT staff
(Rademacher, 1989; Yap et al., 1992) and clear identifica-
tion of project outcomes (Somers and Nelson, 2001; Biehl,
2007).

Although there is now a vast body of literature pertaining
to success factors in systems development contexts, as
noted in this paper’s introduction, there is no significant
evidence to suggest that the adoption of these universal
prescriptions – such as user involvement or senior
management commitment – have led to any noticeable
improvement in project outcomes. Part of the problem may
well be that although the success factors approach has
many attractions, it is also flawed in a number of significant
ways. For example, it has been argued (e.g. Nandhakumar,
1996; Bussen and Myers, 1997; Larsen and Myers, 1999;
Goldfinch, 2007) that:

(i) The success factors literature views system develop-
ment projects as a static process instead of a dynamic
phenomenon, and therefore ignores the potential for a
factor to have varying levels of importance at different
stages of the development and implementation
process. For example, user involvement may be very
important during the systems analysis, testing and
implementation phases of a project, but less so during
the software-coding phase.

(ii) The success factors approach does not explicitly
recognise the variability of systems development
projects, and therefore it fails to account for the
dynamics of the social, organisational and political
context in which any IS project will unfold. For
example, it can be argued that the effects of user
participation on project outcomes may vary greatly
depending upon contextual factors such as: participa-
tion forms; types of participants; participation climate;
and leadership styles (He and King, 2008).

(iii) The approach treats each individual success factor as a
discrete independent variable, and it therefore fails to
take account of any potential inter-relationships
between the variables. For example, the clear identi-
fication of appropriate project outcomes may be
dependent upon active user involvement during the
early stages of an IT investment project.

(iv) The existing literature also typically assumes that these
success factors are purely focused upon a project that
concludes with the delivery of the technical artefact.
More specifically, while the dependent variable may be
composed of operational measures (efficiency, effec-
tiveness), as well as project-related measures (budget,
timescale, etc.), by and large project-related factors are

used to define the independent variable (Nelson, 2007).
Indeed, success factors – such as: project selection;
project team members; project champion; user parti-
cipation in project and project resources are widely
adopted (e.g. Milis and Mercken, 2002; King and
Burgess, 2006; Wang et al., 2008) – are frequently and
explicitly defined in project-related terms. This is an
important limitation of the existing literature, as it
implies that the realisation of benefits is wholly
dependent upon actions that were undertaken during
the systems development project.

A further problem with the factors approach, as noted by
King and Burgess (2006), is that many, if not most, success
factor studies conclude with a list of factors but provide
little further guidance about how and when these factors
should be applied in the context of actual IT projects.
Consequently, all too often there is a serious disconnect
between success factors and project success, so that it
becomes difficult to discern any clear causal relationships.
Finally, in addition to these much rehearsed criticisms, it
can also be argued that as the approach is project focussed,
it typically fails to take account of organisational learning
and capability development over a significant period of
time, in which many individual projects may be undertaken
(Nelson, 2005).

In conclusion, the stream of literature on success factors
in systems development is far too pervasive and substantial
to be completely without merit, yet it appears that the
common prescriptions it offers need to be far better
explained and focused if they are to become more effective.
One potentially fruitful, yet currently unexplored, line
of enquiry is to investigate how success factors might
be re-configured if they were more explicitly focused
towards the delivery of benefits in the medium to long
term, rather than the delivery of a new piece of IT in the
short term.

Benefits-driven approaches to systems development
There has been a great deal of prior academic interest in,
and indeed argument about, how the benefits arising from
IT can best be measured (e.g. Delone and Mclean, 1992;
Farbey et al., 1992; Irani et al., 2007; Petter et al., 2008;
Mitra et al., 2011). Despite this uncertainty, it has also been
argued that benefits should act as the focal point for all
design and ongoing development activity throughout a
system’s operational life (Ward and Elvin, 1999). However,
in practice, it has been found that IT project teams tend to
only consider benefits when writing the initial business
case, but then once approval has been granted, any ongoing
benefits focus tends to rapidly fade away (Ashurst et al.,
2008). What is more, even if organisations did want to use
their business cases as the point of departure for a more
proactive attempt to manage the delivery of benefits, they
would still face two major problems. Firstly, in to get
their projects approved, the writers of business cases
typically overestimate the benefits and understate costs
(Ward et al., 2008). Secondly, as it is widely argued that the
outcomes from systems development projects are typically
emergent, over the life of the system, rather than planned
(Orlikowski, 1996), it is unlikely that the initial business
case will provide a reliable road map for the downstream

The successful realisation of benefits NF Doherty et al
3

management of benefits. Against this backdrop, it has
become clear that there is a great deal more to benefits
realisation than simply encouraging IT professionals to
stick to their business cases.

As benefits rarely spring automatically from the introduc-
tion of a new technology (Eason, 2001; Doherty et al., 2003),
the adoption of a benefits realisation programme is increas-
ingly seen as an important mechanism for proactively
managing new IT initiatives (Peppard et al., 2007; Ashurst
et al., 2008). The defining characteristic of a benefits
realisation approach is that it seeks to facilitate a programme
of organisational change that will complement a new
information system’s functionality, and in so doing facilitate
the realisation of important benefits (Serafeimidis and
Smithson, 2000; Hughes and Scott Morton, 2006; Peppard
et al., 2007). Such technologically mediated organisational
changes come in all shapes and sizes. For example, an
organisation is far more likely to realise benefits from its new
Enterprise Resource Planning (ERP) implementation if it
explicitly redesigns its working practices to more closely
reflect the process model embedded in the system (Ke and
Wei, 2008). Similarly, benefits may only be leveraged from a
new data warehouse if the host organisation actively seeks to
modify its culture so that its staff are more flexible, customer-
focussed and empowered (Markus, 2004).

In addition to its explicit focus on organisational change,
the other really significant area in which benefits reali-
sation differs to more traditional approaches is in terms of
its timescale. Although it may be possible to effectively
plan some types of benefit in advance, others are emergent
as users innovate and improvise with their local working
environments (Orlikowski, 1996); as powerful actors
initiate technologically occasioned organisational change
(Leonardi and Barley, 2010) or as information from a new
system is used to guide further organisational change
(Leonardi, 2007). Indeed, the operational life of a system
is typically punctuated by a series of changes to the
organisational form and function, coupled with modifica-
tions to the information system’s design (Lyytinen and
Newman, 2008). BRM therefore has a critical role to play in
both facilitating and regulating this ongoing process of
organisational change to ensure that opportunities are
exploited in a productive manner (Ashurst et al., 2008).

Unfortunately, despite this growing interest (e.g. Ward
et al., 1996; Reymeni and Sherwood-Smith, 1998), the
benefits realisation agenda is exhibiting many of the same
characteristics, as the socio-technical literature (Mumford,
1995; Avison et al., 1998): an excellent idea, in theory, but
having little impact on the way projects are being managed,
in practice (Pfeffer and Sutton, 2000). There is a growing
body of literature that advances the case for a variety
of different benefits realisation tools, techniques and
approaches (e.g. Reymeni and Sherwood-Smith, 1999;
Bradley, 2006; Ward and Daniel, 2006; Esteves, 2009).
However, there has been relatively little empirical investi-
gation of what, if anything, organisations are doing in
practice, to proactively manage benefits from their IT
investments. Against this backdrop, there is a pressing
need for novel contributions that present insights into how
an explicit focus on benefits realisation might best be
incorporated into the actual routines of systems develop-
ment and implementation.

Critique of literature and research objectives
For far too long IS success has been primarily defined,
and therefore IT project managers rewarded, in terms of
completing a software development project on time, to
specification and within budget (Nelson, 2005; Sauer and
Davis, 2010). Indeed, the most well-used index of informa-
tion success/failure – the biennial Standish Group reports –
defines success using these very criteria (e.g. Standish,
2001). Consequently, the literature on success factors for
IS development has also typically adopted a short-term
perspective, which assumes that the success of projects can
be judged once the software development project has been
completed, which is normally shortly after implementation
(Sauer et al., 2007). In practice, however, the delivery of IS
on time, to specification and within budget, does not
automatically equate to the delivery of real benefits to the
host organisation, as there is increasing recognition that the
impacts of IT are typically emergent, over the operational
life of the system (Orlikowski, 1996; Lyytinen and Newman,
2008), and its benefits cannot therefore be readily or
accurately planned in advance. Consequently, it is appro-
priate to view benefits realisation as an ongoing journey,
rather than a destination (Goh and Kauffman, 2005;
Hardgreaves and Armstrong, 2005).

Against this backdrop, the project’s process-focused
success factors that contribute to the successful outcome
of software development projects might not identify with
those that are necessary to deliver real organisational
benefits in the longer term. The key difference, apart from
timescale, would seem to be that the traditional success
factors literature focuses primarily on the delivery of a
technical artefact, and rather ignores, or underplays, the
need for complementary organisational change, upon
which the realisation of business benefits is dependent
(Markus, 2004). Indeed, project management methodolo-
gies like PRINCE2 focus primarily on delivering ‘the
product’ (i.e. the technical artefact), paying far less
attention to business benefits. Consequently, there is a
pressing need for a critical re-evaluation of the traditional
prescriptions for ensuring the successful outcome of
software development projects, to see whether they take
on a different form when being applied within the confines
of IS development initiatives that have an explicit benefits
realisation orientation. In particular, we were keen to
explore how such success factors might be modified, if their
purpose was to facilitate the realisation of meaningful
business benefits in the long term, rather than the delivery
of a technical artefact in the short term.

Overview of the research methods
The purpose of this section of the paper is to review and
present the methods by which the research data were
collected, validated and ultimately analysed. However,
before reviewing the research methods, it is necessary to
comment on our philosophical perspective (Lee, 1999),
which can be broadly categorised as ‘interpretive’ as our
aim was to gain ‘knowledge of reality’ through the study of
social constructions, in particular, language and documents
(Klein and Myers, 1999). In particular, this study adopted a
dialectic hermeneutic approach (Myers, 1994: 58) to help
make sense of an information system’s development and

The successful realisation of benefits NF Doherty et al
4

utilisation process, in which ‘different stakeholders may
have confused, incomplete, cloudy and often contradictory
views on many issues’. Moreover, in designing this study,
we prioritised the need to produce ‘relevant and timely’
research (Davenport and Markus, 1999: 20) and to ‘produce
knowledge about how to intervene in the world and change
it in to satisfy real-world needs’ (Lee, 1999: 29). The
aim of this section is to provide a review of the context in
which the research was located, before reviewing the overall
research design, and then describing the targeting, execu-
tion and analysis of the case studies.

Research design
To provide rich and critical new insights into the realisation
of benefits from IS development projects, we needed to gain
a high degree of access to IT professionals and business
stakeholders working on a variety of IT development
projects. Public sector organisations were targeted, as we
perceived that they might have the most to gain from
involvement in our research, as prior research suggests that
such organisations have typically struggled with IT projects
(Fountain, 2001; Goldfinch, 2007), and they are generally
more willing to be more open about their experiences than
their private sector counterparts. Ultimately, we gained
permission to conduct in-depth case studies at three public
sector organisations, each of which had three or more
individual IS projects underway, which could be studied.
The decision to focus on a variety of projects within each
case organisation was an important element of the research
design, as it allowed us to focus upon organisation-wide
competences and practices, rather than project-specific
approaches. The first system investigated, in the first case
organisation, was used as a pilot to test out the overall
approach to collecting and analysing evidence.

Data collection
The aim of our study was not to define and explore
hypotheses about the role and impact of benefits realisation

approaches in a ‘positivist’ sense but to develop a rich
understanding and explanation of a highly complex
situation, which in the longer term may be of real use to
practitioners (Eisenhardt, 1989). Consequently, our pri-
mary data collection instrument was the semi-structured
interview, which allowed for a high degree of flexibility, and
at each case organisation we interviewed at least ten
individuals, some of whom we interviewed more than
once. The interviews were either tape-recorded or detailed
notes were recorded, depending upon each interviewee’s
preference. To provide a broader perspective, and to
triangulate the findings, a number of key project events –
such as steering committee or project meetings – were
observed at each organisation, and a variety of project and
strategic documentation were critically analysed. Upon
completion of each data collection exercise, a provisional
analysis of the data was conducted, after which a series of
follow-up meetings were held with each case organisation, to
validate and extend the analysis, as well as helping to fill
any gaps in our understanding. A more detailed review
of the data collection strategies adopted at each of the
three case organisations is presented in Table 1. It can be
seen from an inspection of the data in Table 1 that there
are some imbalances in the number of data collection
activities conducted at each case site, which is inevitable in
circumstances in which access to each data source has to be
individually negotiated with each case organisation. How-
ever, given the relatively modest nature of these differences,
and the large amount of data collected at each site, we do not
believe that such differences will have made any material
difference to our analysis and interpretation of the data.

Data analysis
The notes made during each interview were reviewed and
typed up immediately after the interview, after which
additional ‘marginal notes’ (Miles and Huberman, 1994)
and a brief summary of key themes were added. This
data-recording and preliminary analysis was in line with

Table 1 Data collection approaches

Organisation
type

Organisation A: strategic health
authority

Organisation B: university Organisation C: city council

Interviews/
Interviewees

Seventeen interviews with
fourteen interviewees: business
sponsor, programme manager,
project manager, users, etc.

Seventeen interviews with
seventeen interviewees: Dean;
IT Director; analysts, users, etc.

Eleven interviews with ten
interviewees: Director of business
division; IT Director; project
managers; transformation
manager, etc.

Document
review

Examples include: business
change plan; project initiation
document; health deployment
plan, etc.

Examples include: IS strategy;
requirements specifications;
functional specifications;
project briefs, etc.

Examples include: customer
service strategy; user guides;
communication plan; customer
services plan, etc.

Observation Four individual events: strategy
meetings, benefits workshops, etc.

Seventeen individual events:
strategy, project, progress and
user meetings, etc.

A number of visits were
undertaken to the council call
centre and offices, at which the
researcher could both observe and
informally interview workers

Follow-up
meetings

Four clarification/validation
meetings

Three …