AI on track to ‘crowd out’ human decision-making in warfare

AI on track to ‘crowd out’ human decision-making in warfare

In a briefing at UN Headquarters in New York, UNIDIR Director Robin Geiss had a stark question for journalists: “Are we going to let machines target and kill human beings?” he asked, warning that AI was increasingly “on track to crowding out human decision-making” in the sector. 

The UN Secretary General has been repeatedly urging stakeholders to make sure we do not cross this “moral red line.”

Master and commander

AI is being given a growing number of responsibilities in military affairs, from intelligence analysis and target identification to drone operations and orchestrating cyber-attacks.

Crucially, this comes at the expense of human judgement, including in the nuclear domain, Mr. Geiss stressed. 

The shift is dramatically increasing the pace of battlefield operations, shortening the time between identifying and striking targets and allowing forces to engage an unprecedented number of targets each day. 

At the same time, many of these evolving AI systems are opaque, and the UNIDIR chief warned they are increasing the risk of error, miscalculation and therefore of unintended escalation. 

As AI tools are becoming easier to use, it will also become easier for non-State actors to engage in ever-more sophisticated military operations, he said. 

Geopolitical convergence

Mr. Geiss recognised that geopolitical competition and tensions are “serious obstacles” to regulating AI involvement in the military sector.   

Yet he also stressed that concerns about its risks are increasingly being shared – regardless of political outlook – including among major global powers. 

In September, for example, the United States proposed an AI “notification mechanism” with China, so the world’s two largest military spenders can warn each other about AI incidents that could threaten national security.

The UN Secretary General’s repeated calls for stronger regulation of military AI and a legally binding treaty on lethal autonomous weapon systems are also “increasingly [finding] an echo across the international community,” the UNIDIR Director said. 

A multilateral response 

The UNIDIR is working actively to create some of the building blocks required for multilateral cooperation on the matter. 

On Thursday, the UNIDIR Centre of Excellence on AI, Peace and Security will launch a new multi-stakeholder forum on AI in international peace and security and in the military domain – the first and only such forum within the UN system so far, said Mr. Geiss. 

The forum will focus on three areas:

1. Developing a framework for responsible industry behaviour, with practical, voluntary guidelines for companies to follow at the design and development stage for AI military tools. 

2. Developing an “AI security lexicon” to clarify key concepts such as autonomy, safety or security, which are currently being used very differently by different stakeholders – creating ambiguity that hinders trust and cooperation. 

3. Addressing the risk of misuse of AI by non-State armed groups to support operational planning, execution, and logistics.