Resettlement agencies that oversee the relocation of refugees are increasingly relying on artificial intelligence (AI) systems to aid in their decision-making processes. However, the use of automated decision-making (ADM) systems raises concerns about the potential for discrimination and bias in refugee resettlement.
Of the 10 agencies responsible for refugee resettlement, at least half utilize ADM technology to assist in making placement decisions. One such agency, HIAS, partnered with researchers in 2018 to develop Annie MOORE, one of the first ADM systems to aid in refugee resettlement. Annie MOORE aimed to recommend placements that would maximize employment opportunities for incoming refugees.
However, the developers of Annie MOORE acknowledged that the system’s prediction abilities were limited due to a lack of comprehensive data. Instead, the system primarily relied on short-term employment outcomes of refugees within the first 90 days of their arrival. This approach overlooked crucial factors such as long-term employment prospects, physical and mental health, education, and household earnings, which are essential in measuring the success of refugee resettlement.
Although HIAS has since implemented an updated ADM system called RUTH, the relative scarcity of data remains a concern. The developers of RUTH highlighted that while they have data on hundreds of thousands of refugees, the available information only spans 15 fiscal years and is incomplete and limited in scale for earlier years.
The reliance on ADM systems in refugee resettlement raises questions about their potential to perpetuate inequality and discrimination. A working paper from Harvard Business School published in February 2023 revealed that the impact of existing resettlement algorithms varied significantly depending on a refugee’s age, education, and country of origin. These discrepancies have legal implications, particularly if the use of an ADM system leads to disparate outcomes for refugees from different countries of origin, potentially violating the EU’s General Data Privacy Regulation.
While Congress has yet to pass comprehensive data privacy legislation, the US Code’s Title 8 provides the government with jurisdiction to regulate decision-making processes of resettlement agencies in the event of discrimination based on nationality.
Moreover, case law highlights potential issues that may arise when resettlement agencies rely on ADM systems. In Bauserman v. Unemployment Insurance Agency, the Michigan Supreme Court ruled that the state’s ADM system, which automatically categorized applications as fraudulent, violated the state’s constitution. The court held the agency legally accountable, resulting in a $20 million settlement for individuals wrongly accused of fraud by the system.
Although ADM systems used in refugee resettlement differ from the specific case in Michigan, it underscores the need for resettlement agencies to prioritize due process requirements and avoid discriminatory effects when deploying these technologies.
Resettlement agencies play a crucial role in assisting refugees in their transition to a new country, providing opportunities for them to prosper. While the use of AI and ADM systems can expedite the resettlement process, it is essential to ensure that these systems do not inadvertently disadvantage the very individuals they aim to help.
In conclusion, the increasing reliance on AI in refugee resettlement brings both benefits and risks. While technology can aid in decision-making and streamline processes, the potential for discrimination and unequal outcomes must be carefully addressed. Resettlement agencies must prioritize comprehensive data collection, transparency, and adherence to legal and ethical standards to ensure the fair and successful resettlement of refugees.
