USAID Appropriate Use Framework slides (PDF - 2.1MB)
Learning Objectives
- Present the appropriate use framework developed by USAID.
- Define and be able to apply concepts of relevance, representativeness, value, explainability, auditability, fairness, and accountability/responsibility.
Content
Relevance
- Is the use of ML in this context solving an appropriate problem?
As ML becomes more of a trend, we see more and more organizations apply it to their work to distinguish themselves from competitors or increase their appeal to funders without understanding whether it is the right tool for the problem they are trying to solve.
Representativeness
- Is the data used to train the ML models appropriately selected?
In order to evaluate representativeness, the organizations should consider if the ML model uses data representative of the context in which it will be deployed and which strategies are important for ensuring models can be trained with appropriate data.
Value
- Does the machine learning model produce predictions that are more accurate than alternative methods? Does it explain variation more completely than alternative models?
- Do the predicted values inform human decisions in a meaningful way? Are they actionable? Are they timely? Are they delivered to the right people?
Explainability
- How effectively is the use of machine learning communicated?
It is important to ensure that the application is explained to end-users in a way that effectively communicates how the outcomes were determined. Organizations seeking to apply machine learning outcomes without understanding the nuances of how the models make decisions may use the algorithm outputs inappropriately.
Auditability
- Can the model’s decision-making processes be queried or monitored by external actors?
Increasingly organizations are turning to “black box” machine learning solutions, whose inner-workings can range from unintuitive to incomprehensible. It is important that the outputs can be monitored externally to show that the model is fair, unbiased, and does not harm some users.
This may require additional infrastructure, whether it is an institutional or legal framework that requires audits, provides auditors with secure access to data and algorithms, and requires that people act on their findings.
Fairness
- If used to guide decision-making, has the ML model been tested to determine whether it disproportionately benefits or harms some individuals or groups more than others?
Testing the results of algorithms against protected variables such as gender, race, age, or skin color is key to preventing the adoption of biased algorithms. Does a specific algorithm fail for specific groups of people more often than for other groups of people? Does it misclassify different groups in different directions—or do certain groups have different rates of false positives and false negatives?
It is also important to address the issue that accuracy and fairness are not necessarily correlated. Algorithms can be accurate technically, but still inconsistent with the values that organizations may want to promote when making decisions such as who should be hired, who should receive medical care, or others. Gaining an understanding of how these outcomes are derived and taking steps to mitigate them is an important piece in ensuring that unfair algorithms are not widely adopted and used.
Accountability/Responsibility
- If used to guide decision-making, are there mechanisms in place to ensure that someone will be responsible for responding to feedback and redressing harms, if necessary?
For example, an algorithm might be used to assist in diagnosing medical conditions, but the final diagnosis should still be provided by a trained medical professional. When used by itself, false-identifications from the algorithm could cause harm to individuals. However, consider if there is a shortage of trained medical professionals. Does the risk of misdiagnosis outweigh the risk of not treating people? These decisions are complicated and there is not always a right answer. An accountable setup both makes sure that there are systems in place to prevent harmful errors and makes sure someone is responsible for and correcting errors.
Discussion Questions
- What are examples of ML being applied to irrelevant problems in international development?
- How would you address the issue of representativeness in your data?
- When might it be more important to choose an algorithm that is more explainable at the cost of accuracy or speed?
- How would you implement accountability and responsibility in an existing system?
References
Paul, Amy, Craig Jolley, and Aubra Anthony. “Reflecting the Past, Shaping the Future: Making AI Work for International Development.” USAID, Washington (2018).
Contributions
Content presented by Amit Gandhi (MIT).
The guiding principles presented in this section were developed by Amy Paul, Craig Jolley, and Aubra Anthony of the Strategy & Research team within the Center for Digital Development at USAID. See their USAID report (PDF - 4.7MB).
