2 Executive summary: what you need to know as a manager

“Everything simple is false. Everything which is too complex is unusable.” Paul Valery, Œuvres II, 1942.

2.1 There’s no simple solution to a complex problem

Numbers of humanitarian interventions in protracted context, such as Cash-based Interventions, distribution of core relief items, professional training, micro-credit and other livelihood intervention but also community leaders training, require targeting. In such context, it’s key to have organized tools to first profile the people in need that could be eligible for each type of assistance, and then to prioritize beneficiaries with a neutral and objective tool. Scores uses numerical tools to rank order cases using data integrated into a single value that attempts to measure vulnerability risk.

Scoring systems are designed to better assess population profiles and are powerful tool to standardize decision-making when selecting beneficiaries. Such system improve the accuracy of allowance decisions and make assistance more cost-efficient. Operations sometimes assume that setting up a scoring system is too costly or difficult or that they do not have the kind of data needed to implement it. However, the primary input needed for to calibrate such scores are available in most operations: identified filed experts and a registration system. This documents explains how data from both experts and people in need can be used to develop scoring formula and the ways in which it can be used.

Though setting up such scoring systems also requires specific efforts around:

  • Systems business process, such as referral pathway, needs to be fully integrated, online and provide data in real time.
  • Data needs to be captured digitally in real time, fully integrated, and managed by automated processes. For instance, scores values needs to be updates automatically whenever the data describing the case is changing.
  • Information from the scores should be used across different sectors to drive strategy and support programs design, assistance delivery and risk management. While different types of assistance may have different eligibility and prioritization threshold, the way the score are defined should be consistent between all sectors to ensure consistency in the way programmes are designed.
  • Analytics supports should be available to monitor and update the scoring models on regular basis.

Last but not least, the way the scoring calculation are designed should reflect what the organisation is tying to address.

2.2 Scoring with a protection lens

In UNHCR context, it’s important to define a scoring model that actually reflects UNHCR strategic directions and that can inform all potential type interventions that may require targeting

2.3 The reliability of your scores will reflect the quality of your data

Using scoring system is not about setting up data-driven process but rather about ensuring the human processes are data-informed. It is important to keep in mind that whatever formula is being used, it will alwys comes with both inclusion and exclusion errors. To address this, a series of measure should be set-up:

  • Organise regular expert consultation to ensure that any change in the context can be then reflected in the formula

Computer assisted decision does not replace the individual case management but rather allows to rationalize it. Therefore it always necessary to have a redress and grievance mecanism. Such process can have mult

2.4 Be ready for auditing

A lot of attention is now given to build data responsibility into humanitarian action and this has been reflected in recent IASC guidance, this also covers the issue of data-informed utomated decision making such s vulnerability scoring:

  • Fairness: Algorithms discriminate just like humans do, but at a larger scale. Technology must be informed by ethical and legal considerations. as such the development of the scoring formula should be documented in order to be auditable

  • Diversity: Ensuring different kinds of objects are represented in the output of an algorithmic process.

  • Transparency: Users and regulators must be able to understand how raw data was selected, and what operations were performed during analysis. This implies that the processes used for selection beneficiaries should be documented, for instance using diagram with the standard Business Process Model Notation

  • __Equality: Equality of opportunity and equality of outcomes enforce the similar treatment for similar people, believing the current dissimilarity is the result of past injustice.

  • Data protection: Responsibility by design, managed at all stages of the lifecycle of data-intensive applications.