The China Mail - Can we shut down AI?

One of the co-founders of AI firm Anthropic told the BBC this week he believed kill switches may need to be made mandatory for AI.

But if an AI system goes rogue, is it even possible to shut it down?

- Is there a 'kill switch'? -

"From an AI cybersecurity perspective, the term 'kill switch' refers to the ability to interrupt the execution of an AI agent's action," Nicolas Papernot, a professor at the University of Toronto and a specialist in computer security and artificial intelligence, told AFP.

The person or organisation that deployed the AI agent -- an industry term for a bot that has the potential to operate autonomously -- can stop it by denying it access to the computing power it needs to run, Papernot said.

The situation becomes more complicated if the program has been allowed to act beyond its initial environment.

Papernot envisaged a scenario where AI could be used to spread a computer worm -- a type of self-replicating malware.

"In that case, we would have to simultaneously interrupt the action of all copies the worm made of itself -- device by device by device," he said.

- Can we really switch off an AI? -

Thierry Poibeau, an AI expert at the French National Centre for Scientific Research (CNRS), said the image of a single button that would let us "switch off AI" is misleading.

"You can't really just cut off AI like that," he said.

"There are an enormous number of companies around the world, and all sorts of different software and services. There's no single person who controls AI as a whole."

But it is possible to shut down a specific system when you control the machines and services it depends on.

Hussein Abbass, a computer science professor at the University of New South Wales (UNSW) in Canberra, wrote on LinkedIn that AI systems could be divided into three levels of complexity.

In the first, a centralised AI system entirely under the control of its operator, an emergency shutdown mechanism would be "theoretically and practically possible".

But in his other two "decentralised" scenarios, where AI programs have been given permissions to act on other software, files, machines or even systems, it would be much more difficult to stop everything.

- What are the risks of a kill switch? -

The more AI is integrated into everyday activities, the greater the risk that shutting it down will also paralyse legitimate uses.

Papernot said: "As AI is increasingly embedded in all of our institutions and businesses, and as we automate more and more aspects of daily life using AI, it will be harder to use compute power as a kill switch -- because that will mean all of those other legitimate functions will also grind to a stop."

Stopping an AI system could degrade services that businesses, public administrations, or healthcare institutions already depend on.

The costs and drawbacks of such a move would be "enormous", Papernot added.

And some leading public figures have warned against scaremongering about AI.

US President Donald Trump has dismissed existential fears about AI as a hoax, while Meta CEO Mark Zuckerberg has argued against calls to slow AI development, saying market incentives and legal liability are the best safeguards.

For Jean-Gabriel Ganascia, professor of computer science at Sorbonne University, the debate on the "kill switch" can feed into the idea misleading idea that AI's greatest risk is it developing a will of its own.

Instead, he said, the more concrete risk lies in our growing dependence on such systems.

E.Lau--ThChM