Joint Civil Society Statement: Interventions to mitigate the racially discriminatory impacts of emerging tech including AI
As widespread recent protests have highlighted, racial inequality remains an urgent and devastating issue around the world, and this is as true in the context of technology as it is everywhere else. In fact, it may be more so, as algorithmic technologies based on big data are deployed at previously unimaginable scale, reproducing the discriminatory systems that build and govern them.
The undersigned organizations welcome the publication of the report “Racial discrimination and emerging digital technologies: a human rights analysis,” by Special Rapporteur on contemporary forms of racism, racial discrimination, xenophobia and related intolerance, E. Tendayi Achiume, and wish to underscore the importance and timeliness of a number of the recommendations made therein:
- Technologies that have had or will have significant racially discriminatory impacts should be banned outright.
While incremental regulatory approaches may be appropriate in some contexts, where a technology is demonstrably likely to cause racially discriminatory harm, it should not be deployed until that harm can be prevented. Moreover, certain technologies may always have disparate racial impacts, no matter how much their accuracy can be improved. In the present moment, racially discriminatory technologies include facial and affect recognition technology and so-called predictive analytics. We support Special Rapporteur Achiume’s call for mandatory human rights impact assessments as a prerequisite for the adoption of new technologies. We also believe that where such assessments reveal that a technology has a high likelihood of deleterious racially disparate impacts, states should prevent its use through a ban or moratorium. We join the Special Rapporteur in welcoming recent municipal bans, for example, on the use of facial recognition technology, and encourage national governments to adopt similar policies. Correspondingly, we reiterate our support for states’ imposition of an immediate moratorium on the trade and use of privately developed surveillance tools until such time as states enact appropriate safeguards, and congratulate Special Rapporteur Achiume on joining that call.
- Gender mainstreaming and representation along racial, national and other intersecting identities requires radical improvement at all levels of the tech sector.
The structural racism and discrimination that the report identifies as endemic to the field of technology (just as it is in many if not all facets of societies around the globe) cannot be remedied if the teams conceiving of, building, and promoting technological solutions do not understand and represent the concerns of those who will be impacted by them. When implemented, these demographic changes must be meaningful: past “diversity” and “inclusivity” efforts in the industry have often been mere tokenizations of underrepresented groups. Meaningful improvement will mean significant changes to industry power structures, funding flows and models, cultural changes in the workplace, and reevaluation of existing and future product lines that may be employed to target racially marginalized communities and other vulnerable populations.
- Technologists cannot solve political, social, and economic problems without the input of domain experts and those personally impacted.
The past several decades have been dominated by “techno-chauvinism,” the idea that technology alone can solve social problems. But as Special Rapporteur Achiume rightly notes, no algorithmic model, no matter how “perfect,” will solve for centuries of inequality. For this reason, tech design and development must include domain experts (including those with first-hand experience alongside those with professional or academic expertise) in a consequential way. Such experts cannot be briefly consulted late in a product’s development, or worse, after its negative impacts are already being felt, but integrated into the design process. Furthermore, they must be compensated for their contributions. This is especially important in sensitive sectors such as education, criminal justice, and social services, where past technologies have been adopted in spite of flaws and biases that were facially apparent to experts in the respective field. Moreover, such experts can help to identify how well-meaning technologies may be distorted or misused in practice, exacerbating social inequities and harms rather than actually addressing community needs.
- Access to technology is as urgent an issue of racial discrimination as inequity in the design of technologies themselves.
Digital divides illustrate the intersectionality of discrimination in technology, and the COVID-19 pandemic has thrown them into sharp relief. The global South lags the global North in the digital infrastructure that can be employed to provide remote access to medical care, education, and more, but even within so-called “developed” countries, poor and otherwise marginalized communities lack access to these necessary tools. Digital divides are also present within global South states: we have not reached Sustainable Development Goal target 9(c), to bring all those in least developed countries online by 2020. As societies’ reliance on technological interventions increases over time, these divides will deepen in significance and pose life-or-death threats to people around the world.
- Representative and disaggregated data is a necessary, if not sufficient, condition for racial equity in emerging digital technologies, but it must be collected and managed equitably as well.
We welcome Special Rapporteur Achiume’s call for “States to collect, compile, analyse, disseminate and publish reliable statistical data disaggregated on racial or ethnic grounds.” However, despite important advances in data protection laws globally, adequate standards for non-extractive data collection and governance have not yet been disseminated. The history—and present—of data collection is rife with examples of the exploitation of marginalized populations, and the data that is thus taken is then generally managed in such a way that it can too often be used to further target and victimize. We would welcome, and gladly participate, in an effort to develop such standards, which should include both measures to ensure that data is truly representative and respectful, not reinforcing existing racial and other hierarchies; address the power dynamics between data collectors and those whose data is collected, including around meaningful consent; and provide safeguards to prevent such data from being used by malign actors to further oppress marginalized people.
- States as well as corporations must provide remedies for racial discrimination, including reparations.
The signatories followed with interest Special Rapporteur Achiume’s previous report on the centrality of reparations in remedying the many years of socioeconomic oppression that endured during and continue after colonialism and slavery. In this report, the Special Rapporteur rightly notes that states and companies who have been involved with or overseen the development and deployment of emerging digital technologies that introduced novel instances of racial discrimination or deepened existing inequalities are obligated to participate in remedial processes and provide adequate compensation. Given the interplay between economic privilege and access to technology, as well as the rooms in which it is designed and regulated, we wish to underscore the link between these reports, and to encourage the tech giants whose record profits result at least in part from this inequality to consider acting on their public commitments to non-discrimination. This includes meaningful engagement by states and tech companies to evaluate their role in maintaining racial power structures and taking necessary steps to dismantle them.
AI Now Institute
Association for Progressive Communications
Digital Freedom Fund
Internet Sans Frontières
Rashida Richardson, Visiting Scholar, Rutgers Law School (Camden); Advisory Board Member, Center for Critical Race & Digital Studies
Dr. Charlton McIlwain, New York University Department of Media, Culture, and Communication; Advisory Board Member, Center for Critical Race & Digital Studies
Sarah J. Jackson, Annenberg School for Communication, University of Pennsylvania; Advisory Board Member, Center for Critical Race & Digital Studies
Safiya U. Noble, Ph.D., University of California, Los Angeles (UCLA), Department of Information Studies, Co-Director of the UCLA Center for Critical-Internet Inquiry, Advisory Board Member, Center for Critical Race & Digital Studies
UCLA Center for Critical Internet Inquiry (C2i2)