Professor Forman

Ratio Scale Measures of the Importance of Organizational Objectives

In order to measure and manage risks, it necessary to obtain ratio scale measures of the importance of organizational objectives (shown on the right of the bow-tie diagram below) that incur losses from risk events.

The objectives are typically numerous for Enterprise Risk Management and it is often the case that they are so numerous so as to be arranged in a hierarchy, such as in the NIST cybersecurity example below:

NIST 800-30 Threats to Objectives

The risk of the Event in the middle of the bow-tie diagram above depends on both the consequences of the event on each of the objectives as well as the importance of the objectives. Thus, measuring risk requires measuring or estimating the importance of organizational objectives, which is subjective and dependent on judgments from top management as well as others throughout the organization. But how can this be done?

A pairwise comparison method to produce ratio scale priorities from human judgment has been successfully practiced for many years as part of the Analytic Hierarchy Process (AHP). The method is particularly well suited for estimating the importance of objectives in Enterprise Risk Management and Cyber Risk Management for several reasons.

First, there are likely to be many objectives. Research has shown that if there are more than seven, plus or minus two objectives being evaluated, it is advisable to arrange them in a hierarchy of objectives consisting of seven – plus or minus two -homogenous clusters, each with seven plus or minus two objectives or clusters of objectives. Ratio scale measures for the relative importance of each cluster of objectives and objectives themselves are sufficient to estimate the relative priorities of all objectives in the hierarchy provided the estimates of the importance of elements in each cluster are accurate.

Second, in order for the estimates (derived from human judgments) of the importance of elements in each cluster to be accurate, the elements in the cluster should not differ by more than an order of magnitude or so. This is easy to do when arranging the objectives into a hierarchy of clusters and objectives.

Third, a measurement method that elicits judgments about the relative importance of objectives, must be applicable when some of of the objectives are quantitative, such as short term revenue and some are qualitative, such as reputation. The pairwise verbal comparison method of AHP elicites verbal judgments about the relative importance of objectives, taken two at a time, using the fundamental verbal scale of AHP consisting of the words Equal, Moderate, Strong, Very Strong and Extreme — as well as judgments between each of these. This scale is natural for humans, although ordinal rather than ratio in measurement quality.

Furthermore, while this verbal scale is suitable to evaluating importance of qualitative as well as quantitative objectives, each evaluator might have a different internal feeling about the ordinal numerical values corresponding to the words moderate, strong, very strong or extreme.

Fourth: A mathematical technique consisting of the computation of the normalized principle right hand eigenvector of a matrix of pairwise verbal judgments rises to the challenge of deriving accurate ratio scale measured from verbal ordinal input by incorporating redundant verbal judgments that, as long as there is variety of the importance of elements being compared in a cluster, produces remarkably accurate ratio scale priorities. (Google employed this method to become the leader in pageranking). The pairwise process also includes a measure for the inconsistency of comparisons so that a variety of errors can be detected and corrected if necessary.

To see how this method accurately translates human judgment in ordinal, verbal form to ratio scale priorities, click Area Validation Exercise

Finally, having ratio scale measures for the relative importance of the objectives in each cluster, it is straightforward to derive the relative importance for all of the many objectives of the organization and consequently the risks of each event in a form that will allow an optimal allocation of resources in managing the organization’s risks.

%d bloggers like this: