Ratio Scale Impact Measurement Methods

In order to measure and manage risks, it necessary to obtain ratio scale measures of the both the likelihood and impact of risk events. The likelihood of a risk event (shown on the left of the bow-tie diagram below) depends on the likelihood of sources of the risk event and the likelihood of the risk event given the sources as discussed in Ratio Scale Likelihood Measures. The impact of a risk event (shown on the right of the bow-tie diagram below) depends on the consequences of risk events to organizational objectives as well as the importance of organizational objectives.

Measuring Importance of Objectives

The objectives are typically numerous for Enterprise Risk Management and it is often the case that they are so numerous so as to be arranged in a hierarchy, such as in the NIST cybersecurity example below:

It is important to note that the importance of organizational objectives is subjective and depends on judgments from top management as well as others throughout the organization. The challenge is how to obtain ratio scale measures of the relative importance of organizational objectives, when such measures depend on subjective judgments of decision makers.

Fortunately, a pairwise comparison method to produce ratio scale priorities from human judgment has been successfully practiced for many years as part of the Analytic Hierarchy Process (AHP). The method is particularly well suited for estimating the importance of objectives in Enterprise Risk Management and Cyber Risk Management for several reasons.

First, there are likely to be many objectives. Research has shown that if there are more than seven, plus or minus two objectives being evaluated, it is advisable to arrange them in a hierarchy of objectives consisting of seven – plus or minus two -homogenous clusters, each with seven plus or minus two objectives or clusters of objectives. Ratio scale measures for the relative importance of each cluster of objectives and objectives themselves are sufficient to estimate the relative priorities of all objectives in the hierarchy provided the estimates of the importance of elements in each cluster are accurate.

Second, in order for the estimates (derived from human judgments) of the importance of elements in each cluster to be accurate, the elements in the cluster should not differ by more than an order of magnitude or so. This is easy to do when arranging the objectives into a hierarchy of clusters and objectives.

Third, a measurement method that elicits judgments about the relative importance of objectives, must be applicable when some of of the objectives are quantitative, such as short term revenue and some are qualitative, such as reputation. The pairwise verbal comparison method of AHP elicites verbal judgments about the relative importance of objectives, taken two at a time, using the fundamental verbal scale of AHP consisting of the words Equal, Moderate, Strong, Very Strong and Extreme — as well as judgments between each of these. This scale is natural for humans, although ordinal rather than ratio in measurement quality.

Furthermore, while this verbal scale is suitable to evaluating importance of qualitative as well as quantitative objectives, each evaluator might have a different internal feeling about the ordinal numerical values corresponding to the words moderate, strong, very strong or extreme.

Fourth: A mathematical technique consisting of the computation of the normalized principle right hand eigenvector of a matrix of pairwise verbal judgments rises to the challenge of deriving accurate ratio scale measured from verbal ordinal input by incorporating redundant verbal judgments that, as long as there is variety of the importance of elements being compared in a cluster, produces remarkably accurate ratio scale priorities. (Google employed this method to become the leader in pageranking). The pairwise process also includes a measure for the inconsistency of comparisons so that a variety of errors can be detected and corrected if necessary.

To see how this method accurately translates human judgment in ordinal, verbal form to ratio scale priorities, click Area Validation Exercise

Measuring Consequences of Events on Objectives
Measurement methods used for Ratio Scale Measures of event likelihoods including are also applicable to estimating consequences of events on objectives. See: Ratio Scale Likelihood Measures