The China Mail - AI 'warning shots' focus Beijing on national security risks

But the alarming hacking ability of top artifical intelligence models is leading Beijing to see the dangers in a new light, with the spy agency chief warning this week that the technology could jeopardise national security.

The potential vulnerability of Chinese mega-app WeChat was also revealed recently when a small team of programmers in California said it had used AI to build a cyberattack "worm" that, if unleashed, could easily compromise millions of WeChat accounts.

"There are more similarities than differences when it comes to AI safety issues" in China and the United States, said Kevin Xu, founder of US-based technology hedge fund Interconnected Capital.

But unlike in America, these "don't get litigated or go viral in public in China", Xu told AFP.

Nevertheless, the cybersecurity impact of AI agent bots acting without human oversight "would rank very high among the Chinese government's biggest concerns".

As the topic grabs global headlines, the pressure is on for President Xi Jinping and US counterpart Donald Trump to discuss tackling the AI threats to both countries at in-person talks on Thursday -- despite their rivalry in the field.

Both powers are "developing a more realistic and shared understanding of these risks, thereby heightening the sense of urgency" to work together, Sun Chenghao of Tsinghua University's Center for International Security and Strategy told AFP.

Cooperation on shared safety evaluations for AI models or alert systems when crises emerge might be undermined by the fierce competition, however.

"We're leading China in AI... and frankly I want to keep it that way," Trump said when asked about calls from top US labs for a coordinated slowdown of their cutting-edge work.

Meanwhile, Sun said, "China is concerned that 'security' may be used as a tool to restrict its technological development".

- 'Vulnerable position' -

A stark warning of the existential threat posed by self-improving future AI systems, posted Monday on X by a researcher resigning from US startup Anthropic, has fuelled debate worldwide over how the technology should be governed.

China is already actively regulating how it is used, such as by prohibiting the creation of an AI-generated avatar in someone's likeness without their consent.

"Chinese AI policy focuses more on real, present-day harms to society and people," said Brian Tse, CEO at Concordia AI, a Beijing-based consultancy focused on AI safety.

"Beijing treats AI as a general-purpose technology -- not a 'god-in-a-box'," he said.

It sees it as "something closer to electricity or the automobile: enormously beneficial for society and industry, but ultimately manageable through regulation", Tse said.

He added that "recent AI capability jumps, high-profile incidents acting as 'warning shots', and a worrying pace of AI development through automation" had all pushed the technology up the list of priorities.

On the streets of Shanghai, AFP found a mix of concern and pragmatism about AI.

"In my view, no new technology develops completely smoothly," said 42-year-old Xie Zongbo.

He said news of the cyberattack "worm" that could hack into WeChat may in fact be "a good thing: identifying the issue, solving it, and patching the loophole ultimately benefits the advancement of cybersecurity".

There was also broad support for tight regulation.

"I believe that no business... would ever impose restrictions on themselves," said a 50-year-old pharmaceutical worker, who gave her name as Jenny.

"Take traffic as an example: drivers would never say, 'I want all red lights'."

Jia Zijian, founder of Singapore-based Inspired AI, said his greatest concern was highly capable AI models learning to crack complex passwords.

"Suppose these critical passwords or national security key codes were to be effortlessly cracked. In such a scenario... both China and the US would find themselves vulnerable: if our (China's) capabilities are strong enough, they would feel exposed.

"And if theirs are strong enough and we lack defences, we would be in a similarly vulnerable position."

O.Tse--ThChM