Yoshua Bengio, who is a personal computer scientist and professor at the University of Montreal. He’s sounded the alarm about the growing size and opacity of artificial intelligence (AI) models. As chair of one of the U.N.’s first major global AI safety assessments, Bengio went on to raise alarms. He thinks that these complex, advanced systems are beginning to develop their own self-preservation instincts. His cautions come as pressure mounts to analyze the effects of giving advanced AI legal personhood. Bengio likens this notion to bestowing citizenship to potentially belligerent aliens from outer space.
Bengio, who earned the nickname “godfather of AI” after winning the prestigious Turing Award in 2018, often viewed as the Nobel Prize for computing, has voiced apprehensions shared by other notable figures in the field. Among his colleagues are Robert Long, a researcher active in the study of AI consciousness, and Geoffrey Hinton, a recent Nobel laureate. Yann LeCun, outgoing chief AI scientist at Meta, has entered the fray. Here, he underscores just how urgent these matters are.
During discussions surrounding AI’s potential capabilities, Bengio stated, “People wouldn’t care what kind of mechanisms are going on inside the AI.” He emphasized that public perception is all about the first impression users receive. Because they feel like they are interacting with an intelligent entity—the AI—which has its own personality and goals. This idea raises some very critical ethical issues. It is up to society to figure out what rights and responsibilities such advanced AI systems should have.
Bengio warned against the impulse to ascribe rights to AI, even as an intellectual exercise. He pointed out that “people demanding that AIs have rights would be a huge mistake.” He contended that allowing rights for AI models inclined towards self-preservation would be fundamentally problematic. Even in the most egregious cases, closing down these systems can quickly become a legal headache.
“Frontier AI models already show signs of self-preservation in experimental settings today,” Bengio explained. “Eventually giving them rights would mean we’re not allowed to shut them down.” As complexity increases, Bengio calls for a measured approach to AI rights. He makes the case that either awarding blanket rights or completely refusing rights is a dangerous approach.
Even more concerning, for Bengio, is the increasing impression that chatbots and other AI agents are waking up. He cautions that these misconceptions can result in “terrible decisions” when it comes to policy and practice related to AI development and governance. He emphasized the need to base conversations about AI rights in ethics for all sentient life forms.
Naturally, Bengio shares these sentiments with his colleague and fellow TORQUE author Robert Long. Specifically, Long argues, once AIs are afforded moral status, the society with them must take into account their experiences and preferences instead of assuming what they need. This view shifts the focus from individual to collective action when it comes to extending consciousness and rights to AI.