Researchers at the University of Cambridge, in collaboration with The Alan Turing Institute, Princeton, and Google DeepMind, are working on integrating human uncertainty into machine learning systems. Many artificial intelligence (AI) models fail to consider human error and uncertainty, assuming that humans are always certain and correct in their feedback. However, real-world decision-making involves occasional mistakes and uncertainty.
The team sought to bridge the gap between human behavior and machine learning, aiming to account for uncertainty in AI applications where humans and machines work together. This development could promote trust, reliability, and risk reduction in critical areas such as medical diagnosis.
To investigate this concept, the researchers modified a popular image classification dataset to incorporate human uncertainty. Participants were able to provide feedback and express their level of uncertainty when labeling specific images. The study found that training machine learning systems with uncertain labels improved their performance in handling uncertain feedback. However, it was also observed that the involvement of humans led to a drop in overall system performance. The researchers will present their findings at the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society in Montréal.
Machine learning systems that involve humans in the decision-making process, often known as human-in-the-loop systems, are believed to hold promise in situations where automated models alone are insufficient to make judgments. However, when humans are uncertain, this poses a challenge. Human reasoning is inherently influenced by uncertainty, yet many AI models fail to consider this aspect. While various efforts have been made to address model uncertainty, less attention has been given to human uncertainty.
Katherine Collins, the first author from Cambridge’s Department of Engineering, explains that humans frequently make decisions based on the balance of probabilities. In most cases, making mistakes has no significant consequences. However, in critical applications like medical AI systems, accounting for uncertainty becomes crucial. Collins affirms that many human-AI systems assume humans are always certain of their decisions, which is not reflective of reality.
Matthew Barker, co-author and recent MEng graduate from Gonville and Caius College, Cambridge, highlights the importance of empowering individuals to express uncertainty when working with AI models. Humans often struggle to provide complete confidence, unlike machines that can be trained to do so.
For their study, the researchers utilized widely adopted machine learning datasets related to digit classification, chest X-ray classification, and bird image classification. While they had participants simulate uncertainty for the first two datasets, they asked human participants to indicate their level of certainty when looking at bird images, such as determining whether a bird appeared red or orange. These soft labels provided by humans allowed the researchers to evaluate the impact on the final output. However, they observed a rapid degradation in performance when humans replaced machines in the loop.
The research identifies multiple challenges associated with incorporating humans into machine learning models. The team plans to release their datasets to facilitate further research and integration of uncertainty into machine learning systems.
Collins emphasizes the importance of uncertainty as a form of transparency. Trusting a model versus trusting a human requires understanding when to rely on probabilities and possibilities. Better incorporation of human uncertainty, particularly in applications such as chatbots, could offer a more natural and safe user experience.
While the study raises more questions than it answers, Barker concludes that accounting for human behavior can enhance the trustworthiness and reliability of human-in-the-loop systems.
The research received support from various institutions, including the Cambridge Trust, the Marshall Commission, the Leverhulme Trust, the Gates Cambridge Trust, and the Engineering and Physical Sciences Research Council (EPSRC) as part of UK Research and Innovation (UKRI).
