Stuart Russell -- Lethal Autonomous Weapons Systems
Lethal Autonomous Weapons Systems
Autonomous weapons systems locate, select, and engage targets without human
intervention; they become lethal when those targets include
humans. LAWS might include, for example, armed quadcopters that can
search for and eliminate enemy combatants in a city, but do not
include cruise missiles or remotely piloted drones for which humans
make all targeting decisions.
The artificial intelligence (AI) and robotics communities face an
important ethical decision: whether to support or oppose the
development of lethal autonomous weapons systems (LAWS).
The UN has held four major meetings in Geneva under the auspices of
the Convention on Certain Conventional Weapons, or CCW, to discuss the
possibility of a treaty banning autonomous weapons. There is at
present broad agreement on the need for "meaningful human control"
over selection of targets and decisions to apply deadly force.
Much work remains to be done on refining the necessary definitions
and identifying exactly what should or should not be included in any proposed treaty.
Meanwhile, technology moves on. For example, the DARPA CODE program
(Collaborative Operations in Denied Environments)
is described by its program manager as developing
aerial vehicles that operate "just as wolves hunt in coordinated packs
with minimal communication".
Media, publications, etc.
- Davos considers killer robots, Campaign to Stop Killer Robots, January 24, 2015.
- Appearance on the ED Show, MSNBC, discussing autonomous weapons, February 18, 2015.
- Stuart Russell, Artificial Intelligence: Implications for Autonomous Weapons, address to the United Nations meeting on Lethal Autonomous Weapons Systems, Geneva, April 13, 2015.
- Social Responsibility of Robot Manufacturers and the AI community, podcast interview by Noel Sharkey, International Committee for Robot Arms Control, Geneva, April 14, 2015.
- Rise of robotic killing machines has a cautious world talking, by Brooks Mencher, San Francisco Chronicle, April 23, 2015.
- Second UN meeting on killer robots, by the Campaign to Stop Killer Robots, May 7, 2015.
- Stuart Russell, Take a stand on AI weapons, Nature, 521(7553), May 28, 2015.
- US 'paving the way' to AI warfare, Sky News, May 27, 2015.
- Killer Robots Need Regulation, Expert Warns, by Tanya Lewis, LiveScience, May 27, 2015.
- Scientist warns humans could be defenceless 'against deadly drones' within a decade, ITV News, May 27, 2015.
- Killer Drones With AI Will Leave Humankind 'Utterly Defenceless' Warns Expert, Thomas Tamblyn, Huffington Post, May 28, 2015.
- Killer Drones And Artificial Intelligence Is A Recipe For Disaster For Humans, Rhodi Lee, Tech Times, May 28, 2015.
- Artificial intelligence expert sounds alarm on robots that can kill, Radio National Breakfast, Australian Broadcasting Corporation, May 28, 2015.
- Humans will be left 'defenceless' by killer drones: Flying AI robots pose a serious threat to our lives, expert warns, by Jonathan O'Callaghan, Daily Mail, May 28, 2015.
- Killer robots will leave humans 'utterly defenceless' warns Professor, by Sarah Knapton, Daily Telegraph, May 28, 2015.
- Beyond drone warfare: Prof warns of `automated killing machines', Public Affairs, UC Berkeley, May 28, 2015.
- Killer robots 'would leave humanity defenceless', Michael Rundle, Wired UK, May 28, 2015.
- What Might A Killerbot Arms Race Look Like?, Erik Sofge, Popular Science, May 28, 2015.
- The debate on lethal robots is starting too late, Russell Brandom, The Verge, May 29, 2015.
- La tercera revolucion en el arte de la guerra, Javier Sampedro, El Pais, June 2, 2015.
- A.I. Expert Sounds Alarm Over 'Killer Robots', by Macrina Cooper-White, Huffington Post, June 3, 2015.
- Robotic weapons systems raise ethical concerns, by Peggy Hollinger, Financial Times, June 12, 2015.
- Stuart Russell and almost 20,000 others so far, Autonomous Weapons: an Open Letter from AI & Robotics Researchers, July 28, 2015.
- List of 1,812 news articles about the open letter, in 41 languages, compiled by Robert Pattillo of Human Rights Watch.
- Tech Experts Warn Of Artificial Intelligence Arms Race In Open Letter, interview by Audie Cornish, All Things Considered, National Public Radio, July 28, 2015.
- AI experts call for autonomous weapons to be banned, Article 36 (UK NGO), July 28, 2015.
- Artificial intelligence experts call for ban, Campaign to Stop Killer Robots, July 28, 2015.
- Scientists warn over a possible artificial intelligence arms race, interview by Al Jazeera, July 29, 2015.
- A very serious issue lies behind the "killer robots" mockery, by Karlin Lillington, Irish Times, July 30, 2015.
- Ban autonomous weapons, urge AI experts including Hawking, Musk and Wozniak , by Luke Westaway, CNET, July 27, 2015.
- Elon Musk, Stephen Hawking, and thousands of AI researchers just issued a warning about killer robots, by Rob Price, Business InsiderJuly 27, 2015.
- Hawking, Musk, Wozniak: Ban Artificially Intelligent, Autonomous Weapons in War, by Lauren Walker, Newsweek, July 27, 2015.
- LAWS: An Open Letter from AI & Robotics Experts, by Frank Sauer, International Committee for Robot Arms Control, July 28, 2015.
- Leading Experts Sign an Open Letter Calling For a Ban On AI Weapons, by George Dvorsky, IO9, July 27, 2015.
- Many of the World's Top Tech Experts Want to Ban Super-Intelligent 'Autonomous Weapons', VICE News, July 27, 2015.
- Musk, Hawking, and Chomsky warn of impending robot wars fueled by AI, by James Vincent, The Verge, July 27, 2015.
- Musk, Hawking, Chomsky: Why they want a ban on killer robots, by Jessica Mendoza, Christian Science Monitor, July 27, 2015.
- Musk, Wozniak and Hawking urge ban on warfare AI and autonomous weapons, by Samuel Gibbs, The Guardian, July 27, 2015.
- The Scientists and Technologists Who Want To Keep AI Out of Weapons, by Mike Murphy, Defense One, July 27, 2015.
- US Drone Pilots Are As Skeptical of Autonomy As Are Stephen Hawking and Elon Musk , by Patrick Tucker, Defense One, July 27, 2015.
- CNAS Press Note: The Open Letter on the Dangers of Autonomous Weapons, by Paul Scharre, Center for a New American Security, July 29, 2015.
- Stuart Russell, Max Tegmark, and Toby Walsh, Why We Really Should Ban Autonomous Weapons: A Response [to Ackerman], IEEE Spectrum, August 3, 2015.
- Opposition to autonomous warfare swells to 16,000 signatories:
Artificial intelligence community comes together in unprecedented numbers to call for a ban on AI-controlled weaponry, by Alex Hern, The Guardian, 30 July, 2015.
- A nightmare the world has no cause to invent, main editorial, Financial Times, August 4, 2015.
- UC Berkeley Professor Leads Call for Ban on Autonomous Weapons, interview of Stuart Russell by Michael Krasny, Forum on KQED, August 5, 2015.
- A Ban on Autonomous Killer Robots Is Inevitable, by Erik Sofge, Popular Science, August 5, 2015.
- Stuart Russell, Max Tegmark, and Toby Walsh, Why we really should ban autonomous weapons: A response [to Wallace], Kurzweil AI Blog, August 10, 2015.
- Here's the argument for banning killer robots before we're swarmed by them, by Joel Achenbach, Washington Post, August 21, 2015.
- Additional coverage collected by CNAS:
- Satirical coverage arising from the open letter:
- The Now Show, by Steve Punt and Hugh Dennis, BBC Radio 4, August 4, 2015. Segment starts at 5:50. [Warning: not all content on this recording is appropriate for children!]
- Stuart Russell, Hiroshima et Nagasaki: Des leçons pour le présent et pour l'avenir, Le Monde, August 7, 2015.
- Stuart Russell, Ban lethal autonomous weapons, Boston Globe, September 8, 2015.
- Terminators: Fact or Fiction?, panel discussion with Tom Malinowski (Assistant Secretary of State),
Stuart Russell (UC Berkeley), Paul Scharre (Senior Fellow, Center for a New American Security), Missy Ryan (Washington Post), September 14, 2015.
- Stuart Russell, An Open Letter from Artificial Intelligence and Robotics Researchers, talk at CITRIS Day 2015, October 13, 2015.
- Schrecklich perfekte Krieger (Terribly Perfect Weapons), interview by Piotr Heller, Frankfurter Allgemeine Sonntagszeitung, October 18, 2015.
- Stuart Russell,
Robots in war: the next weapons of mass destruction?,
World Economic Forum, January 17, 2016.
- Stuart Russell and others, What If: Robots Go to War,
panel session moderated by Michael Duffy, World Economic Forum, January 21, 2016. Also on youtube.
- Experts on killer robots at Davos, Campaign to Stop Killer Robots, January 22, 2016.
- How to prepare for killer robots, by Alanna Petroff, CNN, January 21, 2016.
- Davos disruptors: Rogue robots, gene weapons, by Arjun Kharpal, CNBC, January 21, 2016.
- Stop killer robots before it's too late, by Victoria Wollaston, Daily Mail, January 22, 2016.
- Stop killer robots before it is too late, scientists tell Davos forum, by Michel Sailhan, phys.org, January 22, 2016.
- Delegates at Davos held a meeting about the dangers of autonomous 'killer robots', by Doug Bolton, The Independent, January 25, 2016.
- It is Time We Considered a Ban on Autonomous Weapons, by John Birks, The Monitor Daily, January 25, 2016.
- Scientists Urge World to Stop Killer Robots at Davos, Agence France Presse, January 22, 2016.
- , by , , January , 2016.
- , by , , January , 2016.
- Nations Must Address Threat of Robotic Weaponry 'by 2018', by Gwen Ackerman, Bloomberg News, January 21, 2016.
- If killer robots arrive, the Terminator will be the least of our problems, by Matt McFarland, Washington Post, January 25, 2016.
- Report Cites Dangers of Autonomous Weapons, by John Markoff, New York Times, February 28, 2016.
- Autonomous weapons and operational risk, by Paul Scharre, Center for a New American Security, February 28, 2016.
- Veil of secrecy lifted on Pentagon office planning "Avatar" fighters and drone swarms, by Dan Lamothe, Washington Post, March 8, 2016.
- Norway wealth fund's ethics watchdog warns firms not to make killer robots, by Joachim Dagenborg and Gwladys Fouche, Reuters, March 11, 2016.
- Sustaining the Third Offset Strategy in the Next Administration, by Shawn Brimley and Loren DeJonge Schulman, War on the Rocks, March 15, 2016.
- Stuart Russell, "Lethal Autonomous Weapons" (video, slides),
CITRIS Research Exchange, April 6, 2016.
- New Dilemmas in Ethics, Technology, and War, study by the American Academy of Arts and Sciences, 2016.
- Pentagon Turns to Silicon Valley for Edge in Artificial Intelligence, by John Markoff, New York Times, May 11, 2016.
- Why the Future Doesn't Need Us -- Revisited, by Ralph Nader, Huffington Post, August 21, 2016.
- Attack of the Killer Robots, by Sarah Topol, BuzzFeed, August 26, 2016.
- Stuart Russell, letter submitted to US Defense Innovation Board hearing, September 28, 2016.
- The Pentagon's Terminator Conundrum: Robots That Could Kill on Their Own,
By Matthew Rosenberg and John Markoff, New York Times, October 25, 2016
- Formal talks should lead to killer robots ban, Campaign to Stop Killer Robots, December 16, 2016.
- Stuart Russell, Do we want machines to make decisions on killing in war?, interview on BBC Tomorrow's World, June 1, 2017.
- The Israeli Military is Buying Copter Drones With Machine Guns, by Patrick Tucker, Defense One, July 13, 2017.
- Top US general warns against rogue killer robots, by John Bowden, The Hill, July 18, 2017.
- Center for a New American Security announces a new Artificial Intelligence and Global Security Initiative, July 25, 2017.
- Stuart Russell and 40 others, letter to President Obama concerning lethal autonomous weapons systems, April 4, 2016.
- President Obama's reply, January 16, 2017.
- Stuart Russell, AI and Lethal Autonomous Weapons Systems, remarks delivered at the inaugural meeting of the Group of Government Experts, Convention on Certain Conventional Weapons, United Nations, Geneva, November 13, 2017.
- Slaughterbots, short film depicting one future with LAWS. Nov 13, 2017.
- Ban on killer robots urgently needed, say scientists, by Ian Sample, The Guardian, Nov 13, 2017.
- Killer robots are almost a reality and need to be banned, warns leading AI scientist, by Mike Wright, Daily Telegraph, Nov 14, 2017.
- This fictional video about AI-powered weapons makes The Terminator look like a Disney film, by Tristan Greene, TheNextWeb, Nov 13, 2017.
- Killer Robots Must Be Banned But 'Window to Act Is Closing Fast', AI Expert Warns, by Aatif Sulleyman, The Independent, Nov 14, 2017.
- 'Slaughterbots' film shows potential horrors of killer drones, by Matt McFarland, CNN Money, Nov 14, 2017.
- Killer robots: Leading AI scientist warns of an apocalypse, Deccan Chronicle, Nov 16, 2017.
- AI Researcher Warns Skynet Killer-Robots "Easier To Achieve Than Self-Driving Cars", by Tyler Durden, ZeroHedge, Nov 15, 2017.
- Who is Stuart Russell? Berkeley AI professor to screen film at UN calling for ban on 'killer robots', by Immanuel Jotham, International Business Times, Nov 13, 2017.
- Des chercheurs en IA créent une vidéo pour demander l'interdiction des armes autonomes aux Nations Unies, by Pierre-Yves Gerlat, Actu IA, Nov 16, 2017.
- Les robots-tueurs, prochaine arme de destruction massive?, by Matthieu Thibodeau, La Presse (Montreal), Nov 14, 2017.
- Dron asesino alerta sobre los peligros de la inteligencia artificial, El Comercio (Peru), Nov 15, 2017.
- Lethal autonomous weapons, interview by Gene Valaitis, Roundhouse Radio (Vancouver), Nov 17, 2017.
- [The world's scientists ask the UN to ban killer robots], Telediario (Spain), Nov 20, 2017.
- This Horrifying 'Slaughterbot' Video Is The Best Warning Against Autonomous Weapons, by David Nield, Science Alert, November 22, 2017.
- "As much death as you want": UC Berkeley's Stuart Russell on "Slaughterbots", interview by Lucien Crowder, Bulletin of the Atomic Scientists, December 5, 2017.
- Military robots are getting smaller and more capable, by David Hambling, The Economist, Dec 14, 2017. (prepublication version, text only)
- [The Future of War: Robots Killing for $30], by Andreas Mink and Michael Furger, Neue Zurcher Zeitung am Sonntag, December 3, 2017.
- Russia to the United Nations: Don't Try to Stop Us From Building Killer Robots, by Patrick Tucker, Defense One, Nov 21, 2017.
- Stop Killer Robots, by David Canepa, The Daily Journal, Dec 11, 2017.
- San Mateo Supervisor to Introduce Resolution Against "Killer Robots", Fox 40 News, Sacramento, Dec 11, 2017.
- Dark Future: Here's When We'll Have the Autonomous Guard Dogs from Black Mirror>, by Dom Galeon, Futurism, December 31, 2017.
- Stuart Russell, Deep learning, autonomous weapons, and the future of AI, interview by Thomas Gouritin, FrenchWeb.fr, January 22, 2018.
- Stuart Russell, Anthony Aguirre, Ariel Conn, and Max Tegmark,
``Why You Should Fear 'Slaughterbots' -- A Response [to Scharre].
IEEE Spectrum, January 23, 2018.
- German Military Has No Plans to Acquire Robot Weapons -- General, Reuters, February 15, 2018.
- Stuart Russell, The new weapons of mass destruction?, Security Times, February 2018, 40-41.
- US wants first drones that can kill people truly independently,
by David Hambling, New Scientist, March 23, 2018.
- Boycott Threat Terminated 'Killer Robot' Project,
by Jeremy Hsu, Discover Magazine, May 6, 2018.
- How a Pentagon Contract Became an Identity Crisis for Google, by Scott Shane, Cade Metz and Daisuke Wakabayashi, New York Times, May 30, 2018.
- Google Decides Not to Renew a Military AI Contract, by Jeremy Hsu, Discover Magazine, June 6, 2018.
- EU Parliament votes overwhelmingly to proceed towards a ban on autonomous weapons, Sept 13, 2018.
- AI's Dark Side, by Syna Kistler, September 24, 2018.
- Stuart Russell, Anders Fogh Rasmussen, Kenneth Payne, Weapons and Warfare: Will autonomous weapons make the world a safer place?, interviews by Jane van Laar, Nieuwsuur, Dutch public television, 2018.
- Fake news? Lethal effect of micro drones, by Philippe Drapela, Swiss Federal Office for Defence Procurement, Nov 4, 2018.
- China's brightest children are being recruited to develop AI 'killer bots', by Stephen Chen, South China Morning Post, November 7, 2018.
- The US Army wants to turn tanks into AI-powered killing machines, by Justin Rohrlich, Quartz, February 26, 2019.
- Fear & Loathing In AI: How The Army Triggered Fears Of Killer Robots, by Sydney Freedberg, Breaking Defense, March 6, 2019.
- Genocide Swarms & Assassin Drones: The Case For Banning Lethal AI, by Sydney Freedberg, Breaking Defense, March 8, 2019.
- Should We Ban Killer Robots? Can We?, by Sydney Freedberg, Breaking Defense, March 11, 2019.
- Don't Let Robots Pull the Trigger, editorial, Scientific American, March 2019.
- Stuart Russell, Les armes autonomes sont la pire des menaces, interview by Sven Ortoli, Philosophie, March 27, 2019. (Online article)
- Why We Should Ban Lethal Autonomous Weapons, video from autonomousweapons.org.
- Stuart Russell, Lethal Autonomous Weapons, invited briefing to UN Secretary General Guterres and all heads of UN agencies, April 29, 2019.
- Report: Kill the idea of killer robots before they kill us, by Justin Rohrlich, Quartz, May 8, 2019.
- Israel Aircraft Industries advert for Mini-Harpy autonomous weapon.
- Killer robots aren't just science fiction anymore, James McLeod, Financial Post, Jan 25, 2019. Includes quotations from Patrice Caine, Chairman of Thales, committing Thales never to produce autonomous weapons.
- Death by algorithm: The age of killer robots is closer than you think, by Kelsey Piper, Vox, June 21, 2019.
- The US Army is developing AI missiles that find their own targets, by David Hambling, New Scientist, August 14, 2019.
- Watch DARPA test out a swarm of drones: It envisions using 250 drones at once, by Jay Peters, TheVerge, August 9, 2019.
- Amazon, Microsoft, 'putting world at risk of killer AI': study, by Issam Ahmed, Agence France Presse, August 21, 2019.
- Campaign To Stop Killer Robots `Immoral' & `Unethical': Bob Work, interview by Sydney Freedberg, Breaking Defense, August 29, 2019.
- Artificial intelligence and war, The Economist, September 5, 2019.
- Artificial intelligence is changing every aspect of war, The Economist, September 7, 2019.
- Information on Turkey's Kargu autonomous attack quadcopter:
- Kargu: Autonomous Rotary Wing Attack Drone Available for Mass Production , Defence Turkey, June 2018. (Article begins 1/3 of the way down page.)
- STM Kargu product page
- Turkey to start operating kamikaze drones near Syrian border in 2020, EhaMedya, September 12, 2019.
- Turkey Now Has Swarming Suicide Drones It Could Export, by Joseph Trevithick, The Drive, June 18, 2020.
- Letter regarding Kargu from the Campaign to Stop Killer Robots to Turkey's Minister of Foreign Affairs, September 26, 2019.
- The robots are coming, the robots are coming, by Justin Rohrlich, Quartz, October 8, 2019.
- Ronald Arkin, Leslie Kaelbling, Stuart Russell, Dorsa Sadigh, Paul Scharre, Bart Selman, and Toby Walsh,
A Path Towards Reasonable Autonomous Weapons Regulation,
IEEE Spectrum, October, 2019.
- Dystopia - The weapons in the new war, Swedish radio documentary, Oct 30, 2019.
- Killer Robots Aren't Regulated. Yet., by Jonah Kessel, New York Times, December 13, 2019.
- Experimental new drone program can use swarms of up to 250 air and ground vehicles to coordinate military raids, by Michael Thomsen, Daily Mail, January 31, 2020.
- Did Russia Just Concede a Need to Regulate Military AI?, by Samuel Bendett, Defense One, April 25, 2019.
- US Navy robot submarine would be able to kill without human control, by David hambling, New Scientist, March 8, 2020.
- Are Drone Swarms Weapons of Mass Destruction?, by Zachary Kallenborn, Future Warfare Series No. 60, United States Air Force Center for Strategic Deterrence Studies, May 6, 2020.
- Swarms of Mass Destruction: The Case for Declaring Armed and Fully Autonomous Drone Swarms As WMD, by Zachary Kallenborn, Modern War Institute at West Point, May 28, 2020.
- Should Killer Drone Swarms Be Considered Weapons of Mass Destruction?, by Kyle Mizokami, Popular Mechanics, May 29, 2020.
- U.S. Army's New Drone Swarm May Be A Weapon Of Mass Destruction, by David Hambling, Forbes, June 1, 2020.
- Chinese helicopter drones capable of intelligent swarm attacks, by Liu Xuanzun, Global Times, May 9, 2019.
- SecDef: China Is Exporting Killer Robots to the Mideast, by Patrick Tucker, Defense One, November 5, 2019.
- Weaponized Drone Swarms Should Be Declared As "WMD", by Tyler Durden, ZeroHedge, August 29, 2020.
- U.S. To Equip MQ-9 Reaper Drones With Artificial Intelligence, by David Hambling, Forbes, December 11, 2020.
- Indian Army Shows Off Drone Swarm Of Mass Destruction, by David Hambling, Forbes, January 19, 2021.
- Tomorrow's War, by Jake Godin and Zach Toombs, Newsy, February 10, 2021.
- Stuart Russell and Zachary Kallenborn on Drone Swarms and the Riskiest Aspects of Lethal Autonomous Weapons, interview by Lucas Perry, Future of Life Institute, February 25, 2021.
- Decoded: Autonomous Killer Robots, by Ali Mustafa, TRT World, March 16, 2021.
- It is time to negotiate global treaties on artificial intelligence, by John R. Allen and Darrell M. West, Brookings Institute TechTank, March 24, 2021.
- Russia Plans Flying `Minefield' To Counter Drone Attacks, by David Hambling, Forbes, April 20, 2021.
- Are you scared yet, human?, BBC Panorama, May 26, 2021.
- Stuart Russell, Anthony Aguirre, Emilia Javorsky and Max Tegmark,
Lethal Autonomous Weapons Exist; They Must Be Banned,
IEEE Spectrum, June 16, 2021.
- Israel's Drone Swarm Over Gaza Should Worry Everyone, by Zak Kallenborn, Defense One, July 7, 2021.
- Stuart Russell, It's time to ban autonomous killer robots before they become a threat, Financial Times, August 5, 2021.
- Lethal autonomous weapons and World War III: It's not too late to stop the rise of killer robots, by Toby Walsh, The Conversation, August 11, 2021.