The growing use of autonomous and artificial intelligence (AI)-guided weapons on the battlefield prompted the UN and International Committee of the Red Cross (ICRC) on Tuesday to call for a legally binding international agreement governing their manufacture and deployment.

UN Secretary-General António Guterres and ICRC president Mirjana Spoljaric warned of the increasingly rapid development and deployment of advanced autonomous and AI-enabled weapons, noting that even arms manufacturers have sought greater clarity about the legal issues surrounding their use. Scientists and engineers developing such systems have also raised concerns about their risks.

Guterres and Spoljaric warned that humanity is “dangerously close to crossing a moral red line”: the autonomous targeting of humans by machines. The risk is that life-and-death decisions will be made by machines, which may be unable to distinguish non-combatants or handle unanticipated situations, such as when a combatant surrenders or is wounded and is no longer participating in hostilities, or when civilians are present in operational areas. This raises questions about compliance with international humanitarian and human rights law as well as accountability for lethal actions. Additionally, the UN and the ICRC underlined that autonomous weapons are no longer a hypothetical abstraction, as the race toward greater autonomy is already underway and many countries are actively developing advanced AI-driven weaponry.

For this reason, Guterres and Spoljaric called on UN member states to establish a legally binding instrument that sets clear prohibitions and restrictions on lethal autonomous weapons. They stressed that this legal instrument should prohibit two types of autonomous weapons: unpredictable weapons and anti-personnel weapons. The first category includes weapons whose operator cannot predict the outcome of the use of force, either by design or during use, while the second type includes weapons that deliberately target a human being, which an ICRC expert said reduce human life to “data, algorithm, sensor and machine operations.”

The UN and ICRC calls came a few months before the seventh Review Conference of the Convention on Certain Conventional Weapons (CCW), expected to take place in November in Geneva.

The UN chief has previously called for a global ban on autonomous weapon systems, describing them as “morally repugnant” and “politically unacceptable.” Similarly, in 2023, the UN and the ICRC called on political leaders to launch negotiations on a new legally binding agreement on autonomous weapons establishing prohibitions and restrictions. The development of these autonomous weapons is closely linked to broader issues of AI development and governance, given the rapid proliferation of AI-enabled targeting technologies.

Pope Leo XIV has described the escalating development of AI-driven autonomous weapon systems as a threat to human life, while the UN has expressed concern about the negative impacts of the use of AI to construct fully autonomous weaponry, including threats to already fragile global peace.