8 Analytic Hierarchy Process

8.1 Defining weights through expert judgment

Often teams of expert suffer from non-aligned goals, power politics, group dynamics and lack of mutual understanding.

How to build consensus on complex decisions in order to raise sufficient confidence in decision outcomes?

Structured technique for organizing and analyzing complex decisions, based on mathematics and psychology
Comprehensive and rational framework to organize feeling, intuition & logic for structuring group decision making. Rather than prescribing a “correct” decision, the AHP helps decision makers find one that best suits their goal and their understanding of the problem
Comparison is made to define priorities between 2 alternatives based on decision-maker’s feeling of priority dues to importance, preference and likelihood of influence. This breaks down complex decision into small judgement
Developed in the 70’s by Thomas Saaty in Wharton Business School. Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World

Assumes a minimum level of judgment consistency among experts
Limited number of criteria to minimize the number of pairwise comparison estimation
Data should be available for all criteria
Assumes that when new alternatives are added to a decision problem, the ranking of the old alternatives is not changing

List potential criteria that would contribute to vulnerability within the current context. Note that criteria can be grouped togehter using hierarchy.
Establish treehold for criteria so that they can be formulated as simple binary questions

Criteria should be saved in a configuration file using the format here data/criteria.csv.

In this case N*(N-1)/2 pairs to review: 5 criteria -> 10 comparisons

Criteria-Code	Criteria-Level-1-label	Criteria-Level-2-label
age	Age of head of household is above 50
gender	Gender of head of household is female
size	Household size is above 5
needs	Occurrence of Specific needs
assitance	Do not Receive assistance

PS: 6 criteria -> 15 comparison, 7 criteria -> 24 comparisons, 8 criteria -> 28 comparisons, 9 criteria -> 36 comparisons

In this case 6 pairs to review…

Criteria-Code	Criteria-Level-1-label	Criteria-Level-2-label
age	Demography	Age of head of household is above 50
gender	Demography	Gender of head of household is female
size	Demography	Household size is above 5
needs	Occurrence of Specific needs
assitance	Do not Receive assistance

A form will allow to collect from expert priorities between criteria by making a series of judgments based on pairwise comparisons:

Use the script 1-Build-xlsform.R to build a xlsform file based on criteria defined above.

The form can be used within UNHCR Kobo server. Experts can be humanitarian case workers that are used to assess vulnerability. See an example here

Once the selected experts (aka. decision-makers) have filled the online form , data can be exported from UNHCR Kobo server in csv format.

Use the script 2-build-hierarchy.R to build a xlsform file to create the AHP file.

This create the file that format correctly the pairwise preferences of each decision-makers to run the next step.

Calculation done using R statistical language.
Synthesize these judgments to yield a set of overall priorities
Check judgments consistency (consistency ratio)
Compute weights for each expert
Mathematical calculations to convert these judgments to priorities for each of the four criteria

Note: Some proprietary software options are also available but would require a Data Protection Impact Assessment before using them.

Knit the 3-final-report.Rmd to get the report. You can see an example here.

An interactive interface is available to interact with results.

Lack of consistency is often observed
If consistency ratio is above 0.1, then judgement are untrustworthy because they are too close to randomness -> exercise needs to be repeated or abandonned.

Calculate mean relative weight
Apply the vulnerability formula
Collect data on each criteria for all household
Apply weight to each record for the different criteria to get the vulnerability level of each household