Project topics and materials
Get Complete Project Material File(s) Now! »
Chapter 4 Experiences with designing ERM infrastructure
CRM: What I’m trying to do is to provide the parameters that you guys would consider to be your bottom line, the things that are fundamentally important to your business operations… WWM: Well… I know what my ten greatest risks are and this doesn’t change them. Is this being used to try and make them more transparent? An exchange between the CRM and another manager over the CRM’s new Operations Risk Framework (Dialogue 4, p 270) The CRM’s vision for developing Watercare’s Risk Management capabilities was dominated by proposals for improving the quality of the company’s “risk data”. Chapter 4 describes and analyses the CRM’s experiences as he engaged in certain tasks toward the implementation of that vision: (i) redesigning the company’s risk framework and registers, (ii) promoting quantitive risk modelling, and (iii) facilitating strategic risk assessment. A detailed account of the CRM’s work is presented in Appendix VI, which constitutes the main empirical data set for the analysis in this chapter. The main points of argument in the chapter concern the descriptions and explanations of what happened in each case, what those experiences reveal about risk as an object of inquiry, and how those insights, in turn, explain the CRM’s experiences. The chapter also draws lessons about the design of ERM infrastructure from the CRM’s experiences. Empirically, Chapter 4 sets the scene for Chapter 5 by revealing how the CRM was relatively unsuccessful in his efforts to improve the quality of Watercare’s “risk data” throughthe design of ERM infrastructure, as he had initially envisaged, but did achieve success by actively facilitating and guiding inquiry. Analytically, the discussion in Chapter 4 lays the groundwork for that in Chapter 5 by (i) framing up the conventional theoretical distinction between Risk and Uncertainty, (ii) explaining the CRM’s failures as the result of a conflict between expectation and reality over how detailed “risk data” should be, and (iii) explaining the CRM’s success as the product of his alternative facilitative approach.
Redefining Watercare’s risk assessment framework
The CRM perceived that many of the problems with the quality of Watercare’s “risk data” stemmed from problems with the company’s risk assessment framework6. The redefinition of that framework was therefore one of the first tasks which the CRM undertook and one to which he devoted considerable attention. The following discussion explicates the key themes from those experiences. The design rationale
The CRM’s primary objective in redefining Watercare’s risk framework was to improve the quality of the company’s “risk data”. As the CRM used the term, “risk data” referred to the information contained in the corporate risk register, representing the outputs of risk assessments performed by staff. The function of that “risk data” was ultimately communicative, i.e. to communicate perceptions of risk between different stakeholder groups in the organisation (most obviously between staff and management, or as the CRM referred to them, between the “coal face” and the “Board room”). From very early on, the CRM had recognised that the risk framework had a normatively important role to play in providing a common language for this communication of risk (see Chapter 3). At Watercare risk was one of two primary criteria on which capital decisions were based (the other being financial cost). In order for any actor to secure capital funding for a project, he or she was required to represent the need for and benefits of that project in terms of risk. The company’s risk framework ostensibly defined the common language and format for that representation, and was, thus, an obligatory point through which all requests for capital and operational expenditure were required to pass (Latour 1987, 1988). Through this function the risk framework was part of the infrastructure for the governance and internal control of the organisation. That is, the risk framework acted to enforce the maxim that capital funding should only be allocated to projects which contributed to the achievement of the corporate objectives. The CRM expressed this governance role as follows: …I’ve realised that one of the main reasons we use risk data is so that people can make an assessment of business need considering the corporate objectives. So, at least in theory, you can look across the entire enterprise and say “this risk data gives me a basis to decide what to do”. It’s an objective method for determining the priorities based on what the organisation wants to achieve, not what I want to achieve. So, to me, you have to have a consistent and objective method for calculating risk, otherwise you can’t compare across the organisation. The fact that different elements of the business pursued incommensurate objectives (i.e. water supply vs wastewater treatment and disposal), but competed for the same limited pot of capital funds, emphasised the need for the framework to objectively represent the corporate objectives and performance standards:
…what you’re saying is that the corporate objectives will in effect define th relative importance of different things. And that’s something I’m very conscious of at Watercare. If I develop the risk framework using the objectives currentlyunder the Statement of Corporate Intent, then I think it would mean that we would stop spending money on wastewater, which would be completely unpalatable to [the wastewater side of the business]. I mean, in essence I’m saying that all risks will be assessed on the significance to the achievement of corporate objectives, but the objective most directly relevant to wastewater is that there will be “no successful prosecutions under the Resource Management Act”. The problem is that it says “prosecutions”, it doesn’t say “compliance with resource consents”… [and] the probability of us getting successfully prosecuted for wastewater overflows is probably very small. In which case, why spend money on it? I realised, holy shit, if we asked [the Wastewater Treatment Plant manager] to do a realistic assessment of the chance that we will get prosecuted, its very small, and his risks will come out Class 2, and he’ll never get any budget. (Dialogue 8, p 301)During his initial review of Watercare’s risk management function, the CRM developed the opinion that the company’s existing (2003) risk framework was not sufficiently well defined to properly fulfil this boundary infrastructure role. The arbitrary categories and vague criteria within the framework promoted rather than constrained the subjectivity of the risk communication process. To address this, the CRM envisaged that the new risk framework should be, as a far as possible, an objective representation of the company’s objectives and representation, and was, thus, an obligatory point through which all requests for capital and operational expenditure were required to pass (Latour 1987, 1988). Through this function the risk framework was part of the infrastructure for the governance and internal control of the organisation. That is, the risk framework acted to enforce the maxim that capital funding should only be allocated to projects which contributed to the achievement of the corporate objectives. The CRM expressed this governance role as follows: …I’ve realised that one of the main reasons we use risk data is so that people can make an assessment of business need considering the corporate objectives. So, at least in theory, you can look across the entire enterprise and say “this risk data gives me a basis to decide what to do”. It’s an objective method for determining the priorities based on what the organisation wants to achieve, not what I want to achieve. So, to me, you have to have a consistent and objective method for calculating risk, otherwise you can’t compare across the organisation. (Dialogue 8, p 301) The fact that different elements of the business pursued incommensurate objectives (i.e. water supply vs wastewater treatment and disposal), but competed for the same limited pot of capital funds, emphasised the need for the framework to objectively represent the corporate objectives and performance standards: …what you’re saying is that the corporate objectives will in effect define the relative importance of different things. And that’s something I’m very conscious of at Watercare. If I develop the risk framework using the objectives currently under the Statement of Corporate Intent, then I think it would mean that we would stop spending money on wastewater, which would be completely unpalatable to [the wastewater side of the business]. I mean, in essence I’m saying that all risks will be assessed on the significance to the achievement of corporate objectives, but the objective most directly relevant to wastewater is that there will be “no successful prosecutions under the Resource Management Act”. The problem is that it says “prosecutions”, it doesn’t say “compliance with resource consents”… [and] the probability of us getting successfully prosecuted for wastewater overflows is probably very small. In which case, why spend money on it? I realised, holy shit, if we asked [the Wastewater Treatment Plant manager] to do a realistic assessment of the chance that we will get prosecuted, its very small, and his risks will come out Class 2, and he’ll never get any budget. (Dialogue 8, p 301) During his initial review of Watercare’s risk management function, the CRM developed the opinion that the company’s existing (2003) risk framework was not sufficiently well defined to properly fulfil this boundary infrastructure role. The arbitrary categories and vague criteria within the framework promoted rather than constrained the subjectivity of the risk communication process. To address this, the CRM envisaged that the new risk framework should be, as a far as possible, an objective representation of the company’s objectives and performance standards (this rationale is represented in Figure 4.1, overpage). He conceived of an idealised hierarchical prioritisation, cascading from the company’s strategic objectives to the specific performance standards and criteria against which the company was held accountable in each of the various functional contexts of the organisation (see Chapter 3). The CRM referred to this as marrying together the strategic (top-down) and operational (bottom-up) perspectives of the organisation (see Dialogue 7, p 295, & Dialogue 9, p 305), which he felt was essential to the design of a good risk framework. In this regard, the framework would also serve as an external arbiter of the value judgements inherent to the concept of risk, because a rigorously defined framework would embed prior decisions about the relative priorities of objectives (and hence of risk outcomes), rather than leaving those judgements up to the subjective perceptions of individual staff members. The CRM’s approach was therefore to limit the subjectivity of the risk framework in use by seeking clarity and detail in the design of the framework (see Figure 4.1). The art of marrying abstract objectives to concrete practices The CRM initially set out to identify, examine, and distill a considerable volume of documentation in order to identify the primary sources of Watercare’s corporate objectives and performance standards and criteria. This research included: the Local Government Act and the Company Constitution, Watercare’s Statement of Corporate Intent, Annual Report, Asset Management Plan, and Funding Plan, the bulk water and wastewater agreements with the LNOs, the New Zealand Drinking Water Standards and Ministry of Health guidelines on the grading of drinking water supplies, and a review of statutory obligations and compliance penalties in the areas of health and safety, human resources management, resource management (consents), and financial reporting. In aggregate, those documents specified a broad range of objectives at each level of the administrative hierarchy, a plethora of performance standards (both internal and external) to which the organisation was held accountable, and hundreds, even thousands of parameters for measuring every facet of Watercare’s performance. Furthermore, those standards, together with the processes and instruments for measuring and reporting performance, constituted a complex metrological system which was not the product of a co-ordinated, topdown design, but rather reflected the historical and ad hoc development of the company and its technologies and practices, as well as the evolution of the broader industry and regulatory environment in which it operated. The risk assessment framework envisaged by Watercare’s CRM sought to represent that complex metrological system within a highly simplified matrix. Since it is the nature of the task of simplification that something must be left out, the design of that framework necessarily involved decisions about which objectives to represent and their priorities, which performance parameters to use, and how to organise both objectives and parameters into a coherent structure which objectively represented “risk” consequencesn for the organisation.
The CRM discovered, however, that the answer to the question of “which objectives are most important?” was not straight-forward, even for someone with a good general knowledge of the organisation. Indeed, there were different answers depending on who answered the question (see, for instance, the CRM’s engagement with Operations line managers in Dialogue 4, and the analysis of that dialogue in Appendix VI, p 472-474). This is not to say that organisational actors are free to arbitrarily prioritise whatever they want, but rather that in any context there may be multiple legitimate ways of prioritising “what matters most” with respect to objectives and performance. Indeed, the fact that the CRM’s revised risk framework would be just one of at least four different enterprise-level systems for the evaluation and representation of Watercare’s performance developed within the company since 1999 pointed up the multiplicity of ways in which the organisation could be interpreted and represented (see Appendix VI, p 480). The CRM thus acknowledged that the task of designing the risk framework was one of trying to “boil down” the “raft of possible ways you could measure organisational performance” to “a handful of those that matter” (Dialogue 13, p 325). The CRM could not resolve the question of what mattered most on his own; or at least not solely by a detached deconstruction of the statements and criteria contained in corporate and regulatory documents. Rather, the task of sorting through and prioritising the complex system of objectives and performance parameters applicable to the various functional contexts of the organisation required considerable operational knowledge of those contexts. This was revealed in all three of his engagements with the stakeholder groups in the Watercare organisation (these engagements are described in detail in Appendix VI): Through his conversations with various managers from the Asset Management group the CRM discovered that the Planning and Project Management units generally had relatively little impact on the immediate (day-to-day) performance of the Watercare enterprise, but were fundamentally responsible for the company’s long-term performance against its primary legislative objective. In this regard, while project-level risks were, individually, often of relatively minor significance at the enterprise level, the systemic risk posed by consistent under-performance of those business units was considerable. It was this knowledge which emphasised to the CRM the importance of tying project-level risk assessment to the performance of individual project managers. The discussion in the August 2007 meeting with the Operations line managers revealed the complex range of inter-related standards to which the company was required to perform, particularly with respect to the delivery and treatment of reticulated water. For instance, for water supply there were performance standards relating to factors of immediate health significance (P1 Determinands), factors of long-term health significance (P2 Determinands), aesthetic factors (taste and odour), the quality of management systems and personnel (MoH Grading), water flow and pressure, and drought security. Evaluating the relative significance of breaches of these standards involved consideration of the importance of the standard (e.g. P1 vs P2 Determinands), the magnitude and duration of the breach, the number of people affected, control actions that the company might take (e.g. imposing water restrictions, issuing Boil Water Notices), and subsequent outcomes (e.g. people getting sick, media attention, prosecution of company personnel, restructuring of the company). The CRM was only able to sort through this plethora of parameters and structure a coherent framework by drawing on the detailed operational knowledge of the actors at the “coal face”. Through his engagement with Watercare’s General Managers the CRM discovered that, while the Statement of Corporate Intent was officially the public statement of Watercare’s corporate objectives, in practice the SCI was strategically irrelevant. This
was not due to wilfull disregard of the document by Watercare’s General Managers, but was rather a product of their in depth understanding of Watercare’s institutional, regulatory, and physical operating environments. In that context, the SCI was important as a statement of Watercare’s corporate social responsibility, but had little impact on the strategic development of the physical water and wastewater infrastructures. The primary forces in that regard were Growth (changes in demand), Levels of Service (changes to regulatory and contractual requirements), and Renewal (replacement or rehabilitation of aging assets). It was this understanding which led the CRM to refocus his attention on the statements of Watercare’s principal objectives in the Local Government Act and the Company Constitution as the appropriate starting point for deriving his risk framework. The common feature in each of the above engagements was that the CRM elicited important understandings about “what mattered” in terms of performance in each of thefunctional contexts: …from a personal point of view, I’ve always heard that historically the guys who were really effective at running companies were the guys who could walk down to the workers on the shop floor and ask ‘what do you do here, what are your needs, what stops you doing what you need to do?’ Which is what I’ve tried to do, and I guess my job then is to go away and try to link that with the corporate objectives. So I think the value that I add is in understanding what they need at the coalface in relation to the corporate objectives, and being able to put it all together in an efficient and consistent structure (Dialogue 3, p 257) Prior to his engagement with the various stakeholder groups, those understandings were not readily apparent to the CRM as an outsider, even though he already possessed considerable knowledge of Watercare’s corporate objectives and performance standards. Individual actors’ perceptions of significance were shaped by a range of factors specific to their practice contexts, including detailed operational understandings of the technical and organisational systems, the nature of the physical outcomes arising from failure of those systems, expectations about public and political perceptions of performance failures, and expectations about personal accountabilities. These were quite specific understandings about how things worked in each context, and about what was important and what was not, acquired by the various actors through long practical experience, both in their professional fields in general, and in their specific roles within the company. It was only with these understandings that the CRM was subsequently able to sort out which objectives and performance parameters should be represented in the framework and how they should be organised. Through this process the CRM also established connections between the objectives and parameters in the risk framework and existing capabilities for data capture, analysis, and reporting within the company. The CRM believed that such connections justified the use of certain parameters over others because they could be objectively quantified through existing business processes. Dialogue 9, for instance, contains a description, in the CRM’s own words, of how he could have interpreted (“sliced”) the corporate objectives in different ways, and of how he focussed on a particular solution because he wanted to link the framework to Watercare’s existing analytical (i.e. modelling) capabilities: Always in the back of my mind was this thought that we need something that we can model, we need to have a measure of consequence that we can model, and since we already hydraulically model the water networks I wanted that to be a measure of consequence… The way that I’m slicing and dicing the objectives, or rather the way that I’m analysing the organisation will enable us to use computational models to do some of the work.(Dialogue 9, p 304) For the CRM, the possibility of using Watercare’s existing models and data to quantify parameters in the risk framework was an important measure of the degree to which he had succeeded in the task of marrying together the strategic focus with the operational focus (just as it was also an indicator of the degree to which the existing framework failed in this task): I think it comes back… to the fact that I’ve been trying to marry the operational focus with the strategic focus. So I started with the objectives and tried to break them down, but I always thought that at some point I’d love to find that they marry well with what the guys on the shop floor think about. So I’ve talked to them and worked out what’s important to them, the point really being that if we’re doing computational modelling then its probably computational modelling of what they think about, isn’t it? (Dialogue 9, p 305) In effect, the links to various practical tools and capabilities, and the contextual understandings revealed through the CRM’s investigation, constituted objective support for why certain objectives and performance parameters should be represented in the framework instead of others. So, for instance, the existence of certain computational models within the company was indicative that the parameters quantified by those models were sufficiently important to warrant the development of special infrastructure for their calculation. Thus, the company’s existing calculative capabilities, and the managers’ specific knowledge of their functional contexts, subsequently supported and justified the CRM’s framework in the face of counter claims that alternative (i.e. extant) definitions of the risk framework might be better. For the CRM, the process of (re)defining Watercare’s risk assessment framework was thus revealed to be less a detached exercise in deconstructive logic and more an artful process of blending, into an explicit text, certain understandings about what was important and how things were done in particular contexts. The understandings were not the CRM’s but were, rather, those of members of the various communities of practice (general managers, planners, project managers, operations staff) which would ultimately have to use the risk framework. In Dialogue 13 the CRM described this process as follows:
…coming up with that framework is really part art, part science. You need to be able to look at a business and understand it, and at the same time you need to be able to look at your objectives and deconstruct them, and then try to marry thetwo together. You have some paper-based objectives, and then you have real business practices and what you’re trying to do is to bring these two together to achieve an efficient translation of information from the bottom to the top.
(Dialogue 13, p 327) This translation between strategic objectives and operational parameters is reflected in the revised Operations risk framework which the CRM eventually proposed (described in detail in Appendix VI, page 489).
Chapter 1 Introduction
Origins and drivers of ERM and the CRO role
Drivers of ERM uptake
Continuing interest in ERM and the CRO role
The state of the CRO profession
The state of formal representation of CROs
The state of knowledge base support for the CRO role
Where and how this thesis contributes
A Theory of ERM: explaining the effects of ERM implementation
A Theory of Action for the CRO: explaining how to perform the role
The relationship between the two theories
Locating the contributions of this thesis
Contributions in engineering
Organisation of the thesis
Chapter 2 Methodology
On performing reliable knowledge
There is nothing so practical as a good theory
From correspondence to resistance: the objectivity of strong networks
The “pragmatic turn” in the social sciences
So we should all be pragmatists?
Performing the role of Chief Risk Officer: a pragmatic inquiry
Shadowing as a way to fulfil a practical interest (and mitigate risk)
Two projects: collaboratively seeking understanding of practice
Pragmatic hermeneutics in response to strategic uncertainty
Judging the account: workability
Judging the account: warrantability
On how I became a transdisciplinarian
Chapter 3 The CRM’s vision for what ERM should look like
The CRM’s proposals for developing ERM at Watercare
Concerns about Watercare’s “risk data”
Concerns about the existing risk framework (2003)
Concerns about risk communication
The evidence from the CRM’s survey of staff perceptions
A vision for better “risk data”
Comparing the CRM’s proposals against normative models
The CRM’s rationale: supporting and defending decisions with “good risk data”
Discussion: ERM and the construction of rational management in organisations
Chapter 4 Experiences with designing ERM infrastructure
Redefining Watercare’s risk assessment framework
The design rationale
The art of marrying abstract objectives to concrete practices
Dilemmas
Difficulties using the new Project risk register
Specifying risk requires a certain degree of knowledge
Knowledge of risk has to be produced
The risk register as a technology of bureaucracy
Limits to modelling the Enterprise Risk Profile
Facilitating the identification of strategic risks
Sense-making in a context of uncertainty, ambiguity, and vagueness
Performing the role of knowledge facilitator
Discussion
Ontological lessons
Epistemological lessons
The value problem: how much detail?
Chapter 5 Rethinking the CRM’s approach to decision support
Rethinking the role of “risk data” (Dialogues 11 & 12)
Rethinking Risk Management (Dialogue 13)
The CRM rethinks his approach (Dialogue 14)
Discussion: “strategic controller” in question
Reinterpreting Risk Management
The CRO’s dilemma: how to convince actors to adopt more advanced capabilities
The potential to add value as a “strategic advisor”
How should CROs support organisational decision making?
Chapter 6 Framing up decision support strategies for Chief Risk Officers
The status of rational choice theory
Understanding real decision making behaviour
Case 1: the formalised construction of a rational choice
Case 2: putting on a skillful performance
Factors affecting framing and calculative processes
The relationship between agent and context
A typology of problem/decision situations
Extending the typology
Reconceptualising “decision” in performative terms
Framing as constitutive of decision
Sources of uncertainty in decision frames
Conditions for cutting off framing reasonably
Decision support strategies for Chief Risk Officers
Strategy 1: Formatting the rational decision make
Strategy 2: Providing resources to the decision maker
Strategy 3: Helping decision makers to settle their frames
Chapter 7 Conclusions
Conclusions
Contributions
Future Research
Dialogues
Dialogue No. 1 – The Vision
Dialogue No. 2 – Quantifying risk profiles
Dialogue No. 3 – Proposed developments
Dialogue No. 4 – Engaging with stakeholders: Operations Managers
Dialogue No. 5 – Engaging with stakeholders: Project Managers
Dialogue No. 6 – Engaging with stakeholders: Asset Managers
Dialogue No. 7 – Engaging with stakeholders: General Managers
Dialogue No. 8 – Reflecting on the dialogue with the General Managers
Dialogue No. 9 – Justifying the risk framework
Dialogue No. 10 – Feedback on strategic risks
Dialogue No. 11 – The purpose of “risk data”
Dialogue No. 12 – On the value of the risk framework in a mature organisation
Dialogue No. 13 – Trying to explain the role
Dialogue No. 14 – Reflections on a year in the role
