- Introduction
This past quarter has witnessed a stirring advancement in artificial intelligence (AI) and a radical reorientation of the discourse surrounding it. Starting at least with the United Nations Economic and Social Council (UNESCO) Recommendations on the Ethical Use of AI, spanning the ‘Encyclical on AI’ by His Holiness Pope Leo XIV and most recently the report of the UN’s Independent International Scientific Panel last week, reflect (among other things), on how advances in AI affect our daily lives.
As AI advances, even the net savvy and digitally suave find it difficult to defend themselves. Last year, a seasoned financial director almost lost half a million USD to a deepfake scam, where the fraudsters successfully impersonated the CEO and got the director to move large sums to them immediately. These are one set of circumstances where AI is used overtly to scam people of money. However, AI also covertly interacts and affects our decision-making, a theme that does not attract enough attention but is a pressing customer protection concern. It is worth asking if our decision-making is not independent of our environment, but is actually shaped by our context, the cues around us, and is dependent on not only the information available to us but also the manners and channel providing that information. As such, we need to ask how AI systems are embellished in choice environments affecting human-decision making.
Put differently, how do we ensure our customers are making informed decisions and not being steered away by optimizing algorithms (that may focus on (sales/revenue/profit)? In essence, how do we resist the tendency of algorithms to dehumanize customers and reduce them to a client to be acquired, a loan to be underwritten or an agent to be manipulated into buying the next financial product?
- What is meant by the de-humanization of people by technology?
This blog defines ‘human’ as the beneficiary, customer, or any individual who is subject to decisions by algorithmic systems. In this context, de-humanization can be defined as:
- the removal or de-prioritization of the ‘human’ from conversations on technology, by instead focusing on other benefits such as efficiency, accuracy, and business-related profits, or it can also include
- the act of perceiving and treating people as if they are less than fully human including through the experience of being subjected to acts that express lack of perception of one’s humanity or the denial of human rights.
More recently, the ability to think for ourselves and mental cognition are being recognized as important facets of human rights that are increasingly at risk from AI and other technological advancements.[1]
De-humanization takes away from the basic tenets of human rights that have been established and reiterated over the years, including the recent development of data rights. Designers and users of technology enable de-humanization of people at the stage of design and deployment of systems.
Virginia Eubanks in her book Automating Inequality explains how the creation of ‘digital poorhouses’ through decision-making systems is one of the first instances of de-humanization of people. Decision-making systems that were used to decide on the delivery of social welfare benefits and access to other basic resources were given the benefit of doubt by human authorities. If these systems decided that a data subject was not worthy of being recognized as human – for instance, because of their inability to provide sufficient documentation to prove they were eligible beneficiaries – the data subject was automatically denied their humanity. They were deemed to be ‘less than human’, removed from the database of the system and any conversations surrounding it, and were pushed into a ‘digital poorhouse’. According to Eubanks, a ‘digital poorhouse’ is a network of vulnerable and excluded people whose lives are automatically subject to the surveillance and scrutiny of technology. She also explains how de-humanization by virtue of being pushed into the digital poor house is not a one-time process. Once a person’s records reflect this classification of de-humanization by technology, it serves as a continuous record for the person’s future.
Critically, an important facet of de-humanization by technology is the inability of people to contest decisions made by automated decision systems. In a related point, de-humanization also includes the inability of designers and users of automated decision systems to explain decisions by systems, including the data and logic used by the algorithm to make decisions.
In the next section, we will explore how systems can be re-humanized through the introduction of rights for people to contest and ask for explanations from automated decision systems.
- The importance of re-humanizing technology: making sense of decisions through explanations
3.1 What is meant by the re-humanization of technology?
Re-humanization of technology is prioritization of rights-based components in the design and use of technology through the treatment of humans as critical and creative agents in socio-technical processes. By critical and creative agents, we mean shifting the narrative away from looking at technology use as a one-way process and instead making it a two-way process. This means shifting the focus of conversations from merely looking at efficiency and accuracy benefits to considering risks that decision-making systems have on human rights. It also means creating an enabling environment for people to exercise their human rights against any unjust decisions, thereby protecting their dignity, autonomy, and agency. Contesting and asking for explanations about the logic of decisions is one way by which technology could be re-humanized. The humane use of algorithms also requires an open and transparent engagement with algorithmic decision-making systems.
3.2 What is an explanation?
Discussions and practices around providing explanations for the working of technology and decision systems began in the 1970s. Expert systems used to make medical diagnosis were designed in a way to offer users transparency and reliability about decision rules. In 2010, however, the use of complex statistical techniques and deep learning mechanisms gave rise to the black box problem and the transparency of decision rules gradually decreased. Scholars began to push back against this black box problem by arguing for the accountability and transparency of decision rules by machines. This push back came in the face of human rights harms that came because of the use of decision systems. Some pivotal incidents that pushed for explanations by automated algorithmic decision-systems included the discrimination in the form of racism, targeting and arrest of people from black communities by police officers and authorities in the criminal justice system in the US. These incidents forced authorities to understand the nuances of decision-making systems, including questions of whether the training data used to make decisions was biased against certain communities thereby enabling discrimination against them along with other rights violations and real-life harms.
3.3 How could the ability to contest for an explanation enable the re-humanization of technology?
An explanation may help people understand why a decision came to be, thereby enabling them to identify errors or grounds to challenge this decision as critical and creative co-agent in the tech ecosystem. Having a clear understanding of the what, why, and how of decisions that people are subject to – specifically decisions by technology – is critical for the re-humanization of technology. Providing an opportunity to contest decisions and ask for explanations is a practical application of the natural principle of audi alterem partem, a latin legal principle meaning “hear the other side”. Audi alterem partem allows for every party in a dispute to be given a chance to represent their arguments, and as such ensures fairness and impartiality. In the context of re-humanization through explanation, this principle can be applied by giving people the option to contest decisions by systems and asking designers and users to make sense of these decisions.
As such, making algorithmic decisions contestable and demanding an explanation is a crucial part of safeguarding human rights and the principles of natural justice. Re-humanization through explanations is best included both at a technical level and policy level. At a technical level, the system be designed in a manner that enables designers to make sense of the decisions, understand what data was inputted, and what logic was used to make the decisions. These safeguards must be made critical, especially complex, unsupervised systems – systems where designers excuse themselves from providing explanation because they cannot make sense of the process. At a policy level, the right to contest decisions or the right to ask for human intervention to challenge decisions and ask for explanations to be provided must be an option given to the human. By providing means to challenge a decision and ask for an explanation, systems:
- respect the dignity of a human by giving them the right to challenge an unjust decision by an automated system,
- agency by enabling them with the choice to challenge the decision by an automated system, and
- autonomy by providing them with the opportunity to decide how their data is used.
As such, explanations and the right to contest may be one of the many ways by which systems can be re-humanized.
- Conclusion:
Explanations for technological decisions serve as a reminder that technological decisions are not a one-way process, and humans are not inanimate beings to be treated as data, numbers or statistics. The GDPR has attempted to humanize solely automated decisions by giving data subjects the right to ask for human intervention in case of an unjust decision. While this is a commendable provision on paper, it may be difficult to put it into practice. Some difficulties include getting support to exercise the right to a human intervention and the right to contest unjust decisions, the ability of designers and users to provide individual explanations on a case-by-case basis, or deciding what to give priority to – the logic/rules applied by the algorithmic system or human discretion. Additionally, designers and users hide behind the complexity of automated systems when asked to provide an explanation about a decision by the system. These complexities could be dealt with if automated systems are designed by centering human interests – including transparency by way of being able to simplify the working of the system and providing explanations about decisions. Attempts to re-humanize technology through provisions of the right to contest, and explanations of decisions do not just fall on researchers, but on designers and users as well. Conversations for re-humanization are imperative and critical and must be prioritized immediately, especially considering the pace and scale at which technology is impacting our everyday lives.
Footnote:
[1] Chile is the first country in the world to amend its Constitution to protect “neurorights”—legal protections specifically designed for brain-related data and mental integrity. Passed in 2021, the reform safeguards mental privacy, personal identity, and free will from advanced neurotechnologies and brain-computer interfaces. [1, 2, 3, 4]
Key Pillars of Chilean Neurorights
- Mental Privacy: Personal brain data is treated with the same legal status as an organ; it cannot be bought, sold, or trafficked. [1, 2, 3]
- Personal Identity & Free Will: Protects individuals from having their brain activity or psychological identity externally manipulated without consent. [1]
- Physical & Mental Integrity: Dictates that neurotechnology and scientific developments must respect the basic psychic and physical identity of citizens.
