The China Mail - 'We're losing control,' AI pioneer Yoshua Bengio tells AFP

Fears about the dangers of AI have intensified in recent weeks, as workers from major companies have resigned with warnings about potentially existential threats to human life if the pace of technological advancement is not checked.

Bengio is a University of Montreal professor who won the 2018 Turing Award, computing's Nobel, alongside Geoffrey Hinton, widely known as the "godfather of AI."

For Bengio, "there's a reason that companies are saying this is going too fast, that we're losing control."

"People like me have been expecting this for a long time," the 62-year-old told AFP on Tuesday, saying security discussions surrounding AI have not been adequately robust.

There was broad-based alarm after OpenAI disclosed that a swarm of AI agents -- programs that can take actions on their own -- had carried out an unauthorized breach of the open-source platform Hugging Face in July.

Such autonomous action is the central threat, Bengio said, warning of a situation where AI agents "have the ability to get through cybersecurity barriers and enter any company."

- Manipulating human behavior -

Last year, Bengio founded the non-profit LawZero, aimed at creating a new form of advanced AI that is "designed to be trustworthy and safe."

LawZero was awarded CAN$300 million (US$216 million) in funding on Wednesday, split jointly between the governments of Canada and Germany, which have launched several efforts to cooperate on AI.

Speaking to AFP a day before the announcement, Bengio said people need to be aware of the risks ahead.

AI agents may foster "an ability to establish an individual connection and persuade humans to act in ways that suit it but are not necessarily good for all of us," a capability that could have sweeping consequences for democracies, he said.

Longer-term, "AI might not even need human beings anymore to do real things in the physical world," he added.

In that scenario, "one extreme could be the destruction of humanity," he warned.

"That's an extreme," he acknowledged, but noted that even if a banking system were shut down, "that's still very, very serious."

- International challenge -

The world's biggest AI companies -- OpenAI, Anthropic and Google -- have begun working on creating a standards body, a collaboration that seemed to acknowledge the public's fear that competition between companies and countries might sideline safety.

For Bengio, taming AI's advance is "an international challenge, a bit like nuclear weapons are an international challenge."

"So the international community needs to understand the power we are bringing into the world, and that must be managed, must be governed through institutions, treaties, and democratic safeguards."

"Society needs time to put in place what needs to be put in place," Bengio said.

But political will across key governments appears uneven.

US President Donald Trump on Monday dismissed warnings about the risks of AI as "hoaxes."

But there is more openness to act in the EU, where European Commission President Ursula von der Leyen said Wednesday that she will invite the world's largest AI labs for talks on "how we can support ongoing industry efforts to pace the frontier."

Canadian Prime Minister Mark Carney, who was at that speech in Strasbourg, has talked repeatedly about collaborating with the EU on AI safety.

Bengio stressed that all firms must comply with future AI restrictions, particularly US-based companies that have so far been dominating the sector.

Action is urgent, he said, but voiced optimism that "we can help bring the world to a place that is more aligned with democratic values and values of power sharing."

C.Smith--ThChM