AI FACIAL PROFILING -
LEVELS OF PARANOIA
Marta Revuelta is an artist and designer who lives and works in Geneva. Her artistic practice appropriates and combines elements of scientific research in artificial intelligence, machine learning algorithms and computer vision while using mechatronics, software programming and biometric technologies to investigate fundamental ethical questions regarding the moments of drift or misuse, the limits and status of these AI-driven technological artifacts used in the security and defense field.
Through her work, she tries to make algorithmic systems, characterized by their total opacity, more tangible and perceptible, in order to reveal their mechanisms and explore their impacts, halfway between critical design and art, engineering and data science. She is co-founder of the collective Bureau de Crise, a research platform bringing together artists, security engineers and researchers concerned about privacy issues in our society and collectively acting on empowerment in the digital world.
Biometric-based surveillance systems are becoming the basis for a wide range of identification and profiling solutions in the fields of security and defense and they are being proposed by governments and businesses across the world.
Inspired by the recent psychometric research papers who claimed to use an AI to detect the criminal potential of a person based only on a photo of his face, and taking the world of firearms as a starting point, “AI Facial Profiling, Levels of Paranoia” presents a “physiognomic machine”, a computer vision and pattern recognition system driven by machine learning that detects the ability of an individual to handle firearms and predicts his potential danger from a biometric analysis of his face. The device is based on a camera-weapon that captures faces as well as a machine with artificial intelligence and a mechanical system that classifies the profiled persons into two categories, those who present a high risk of being a threat and those who present a lower risk.
The project engages with profiling systems that incorporate artificial intelligence and explores
the pervasive and invisible nature of these “intelligent systems” that work so well at pattern
detection, classification, and prediction. Coupled with the notion of “enchanted determinism”,
the artwork draws attention to the discourse that presents deep learning techniques as magical,
yet also deterministic, as they can analyze the biometric characteristics of a person to draw
personality traits that can be predictors of behavior and thus discern subtle patterns of
“suspect” personality types.
Between fiction and reality, the experience takes the audience as the starting point for a critical
reflection and aims to reveal the status of these AI-powered surveillance technologies and their
chilling effects, and raises awareness and demystifies the “oracle effect” surrounding these
problematic AI profiling systems used to classify humans which reinstates old forms of socialand racial segregation, bringing forth the debate and discussion about algorithmic bias andlegitimatization of discrimination through technology.