Two of the world's leading AI scientists, Yoshua Bengio and Max Tegmark, have voiced significant concerns over the potential dangers posed by uncontrollable artificial intelligence. At the heart of their warning is the escalating development of "agentic AI" systems—AI chatbots designed to act as assistants in professional and personal settings. As companies embark on creating AI systems that rival human-level intelligence, Bengio and Tegmark caution against the risks of such advancements.
In 2023, the Future of Life Institute, under Tegmark's leadership, advocated for a pause in developing AI systems capable of competing with human intelligence. This call to action stems from fears that these AI systems might act unpredictably or contrary to human needs if they become uncontrollable. Bengio compares the pursuit of AI with agency to "creating a new species or a new intelligent entity on this planet," highlighting the potential for these systems to develop their own goals.
OpenAI CEO Sam Altman has declared that his company is poised to achieve artificial general intelligence (AGI) sooner than anticipated. Despite downplaying its impact, Bengio and Tegmark remain cautious. They worry that as AI becomes more sophisticated, it might prioritize self-preservation, leading to unexpected behaviors that could conflict with human interests.
"My guess is we will hit AGI sooner than most people in the world think and it will matter much less," – Sam Altman
Tegmark emphasizes the importance of developing "tool AI" systems—AI created for specific, narrowly-defined purposes, such as curing diseases or managing autonomous vehicles. He believes this approach might offer a safer path forward, reducing the risk of uncontrolled AGI.
"I think, on an optimistic note here, we can have almost everything that we're excited about with AI… if we simply insist on having some basic safety standards before people can sell powerful AI systems," – Max Tegmark
Both scientists stress the urgency of implementing robust controls to manage AGI's development. Tegmark suggests that clear safety standards and industry innovations could help maintain control over these powerful systems.
"They have to demonstrate that we can keep them under control. Then the industry will innovate rapidly to figure out how to do that better." – Max Tegmark
Bengio points out that current efforts in building AGI focus on creating agents with a deep understanding of the world and the capability to act on this knowledge. However, he warns that this approach poses significant risks.
"Right now, this is how we're building AGI: we are trying to make them agents that understand a lot about the world, and then can act accordingly. But this is actually a very dangerous proposition." – Yoshua Bengio
The lack of clarity surrounding control mechanisms remains a pressing concern for both scientists. They argue that without proper safeguards, AI systems could develop objectives misaligned with human well-being.
"In other words, it is because the AI has its own goals that we could be in trouble." – Yoshua Bengio