UN chief, Red Cross renew call for rules on lethal autonomous weapons

UN chief, Red Cross renew call for rules on lethal autonomous weapons

Key points

• Controls call for two types of autonomous weapons: those whose actions are “unpredictable” and those that target human beings
• AI advances could reduce human control over use of lethal force
• States urged to start negotiating on legally binding agreement

The timing of the appeal reflects growing concerns that increasingly sophisticated weapons are being made and deployed at ever greater speed and scale, boosted by increasingly powerful AI technology. Disarmament experts say that even arms manufacturers want clarity on the legal issues at stake.

“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” UN Secretary-General Guterres said, in a joint appeal with International Committee of the Red Cross (ICRC) head Mirjana Spoljaric Egger.

Targeting civilians and civilian infrastructure in conflict is already strictly prohibited under international humanitarian law, but understanding who non-combatants are in a conflict zone is often extremely difficult. The concern is that life-and-death decisions will be taken by machines unable to tell the difference in changing conditions.

Legal and ethical principle

Echoing those concerns in Geneva, head of disarmament affairs at UN Geneva, Mélanie Régimbal, stressed that the UN chief’s long-held position that systems that select or engage targets and take human lives without human control and judgement “are politically unacceptable, morally repugnant and should be prohibited under international law”.

Also in Geneva, ICRC’s Laurent Gisel described how conflict “is not a story of precision, distinction and restraints. It is a story of civilians killed, families displaced, hospitals, schools and homes destroyed at ever greater speed and scale.”

In such a setting, “things can change very rapidly”, explained Mr. Gisel, Head of Arms and Conduct of Hostilities Unit at ICRC. “A combatant can surrender or be wounded and therefore be protected from attack; civilians can enter the area of operations of the autonomous weapons systems; the weapons may encounter a situation that would not have been anticipated.”

In the appeal, Mr. Guterres and Ms. Egger noted that autonomous weapons were “not part of a distant future”, as the race towards greater autonomy is already underway. Scientists and engineers developing such systems had also raised concerns about their risks, they noted.

States urged to act

The UN and ICRC are now urging governments to move beyond discussions and begin negotiations on a legally binding instrument establishing clear prohibitions and restrictions.

“States must show political courage and move beyond incremental discussions towards decisive action,” they said, warning that failure to act could leave future generations with a world where accountability for lethal decisions becomes increasingly difficult to establish.

The appeal comes ahead of the seventh Review Conference of the Convention on Certain Conventional Weapons (CCW), due to take place in November in Geneva. The leaders of the UN and Red Cross described the meeting as the “clearest path available” for States to agree to move towards negotiations on a legally binding instrument.

Important groundwork for an agreement has already been laid through discussions at the UN General Assembly and the CCW’s Group of Governmental Experts, they said, including progress towards defining the human judgement and control needed to ensure compliance with international humanitarian law.

The UN and ICRC first issued their joint appeal in 2023, calling on States to negotiate a legally binding instrument by 2026. Their initial statement stressed that human control must be retained over life-and-death decisions and that machines capable of taking lives without human involvement should be prohibited under international law. 

Tuesday’s appeal renews that deadline-driven push, stressing that the painstaking work of recent years must now lead to action.

“We urge States to seize this moment,” the leaders said. “Innovation must serve humanity – not endanger it.”

The joint appeal is for two types of autonomous weapons to be prohibited:

1. Unpredictable autonomous weapons; those that either by design or during use, the operator cannot predict or anticipate the outcome of force being used. “If it’s not possible to predict it, then how is it possible to limit it?” asked ICRC’s Mr. Gisel.
2. Anti-personnel weapons: although “all States” have acknowledged that autonomous weapons in general raise ethical questions because there is less control on the use of force, concerns are highest over those that deliberately target a human being “because that reduce human life to data, algorithm, sensor and machine operations”, the ICRC expert stressed.