The Trouble with Killer Robots

Why we need to ban fully autonomous weapons systems, before it's too late.

Imagine a mother who sees her children playing with toy guns as a military force approaches their village. Terrified, she sprints toward the scene, yelling at them to hurry home. A human soldier would recognize her fear and realize that her actions are harmless. A robot, unable to understand human intentions, would observe only figures, guns, and rapid movement. While the human soldier would probably hold fire, the robot might shoot the woman and her children.

Despite such obvious risks to civilians, militaries are already planning for a day when sentry robots stand guard at borders, ready to identify intruders and to kill them, without an order from a human soldier. Unmanned aircraft, controlled only by pre-programmed algorithms, might carry up to 4,500 pounds of bombs that they could drop without real time authorization from commanders.

While fully autonomous robot weapons don't exist yet, precursors have been deployed or are in development stages. So far, these precursors still rely on human decision making, but experts expect them to be able to choose targets and fire without human intervention within 20 to 30 years. Crude models could potentially be available much sooner. If the move toward increased weapons autonomy continues, images of war from science fiction could become more science than fiction.

Replacing human soldiers with "killer robots" might save military lives, but at the cost of making war even more deadly for civilians. To preempt this situation, governments should adopt an international prohibition on the development, production, and use of fully autonomous weapons. These weapons should be stopped before they appear in national arsenals and in combat.

Fully autonomous weapons would be unable to comply with the basic principles of international humanitarian law -- distinction, proportionality, and military necessity -- because they would lack human qualities and judgment.

Distinguishing between combatant and noncombatant, a cornerstone of international humanitarian law, has already become increasingly difficult in wars in which insurgents blend in with the civilian population. In the absence of uniforms or clear battle lines, the only way to determine a person's intentions is to interpret his or her conduct, making human judgment all the more important.

Killer robots also promise to remove another safeguard for civilians: human emotion. While proponents contend that fully autonomous weapons would be less likely to commit atrocities because fear and anger wouldn't drive their actions, emotions are actually a powerful check on the killing of civilians. Human soldiers can show compassion for other humans. Robots can't. In fact, from the perspective of a dictator, fully autonomous weapons would be the perfect tool of repression, removing the possibility that human soldiers might rebel if ordered to fire on their own people. Rather than irrational influences and obstacles to reason, emotions can be central to restraint in war.

Fully autonomous weapons would also cloud accountability in war. Without a human pulling the trigger, it's not clear who would be responsible when such a weapon kills or injures a civilian, as is bound to happen. A commander is legally responsible for subordinates' actions only if he or she fails to prevent or punish a foreseeable war crime. Since fully autonomous weapons would be, by definition, out of the control of their operators, it's hard to see how the deploying commander could be held responsible. Meanwhile, the programmer and manufacturer would escape liability unless they intentionally designed or produced a flawed robot. This accountability gap would undercut the ability to deter violence against civilians and would also impede civilians' ability to seek recourse for wrongs suffered.

Despite these humanitarian concerns, military policy documents, especially in the United States, reflect the move toward increasing autonomy of weapons systems. U.S. Department of Defense roadmaps for development in ground, air, and underwater systems all discuss full autonomy. According to a 2011 DOD roadmap for ground systems, for example, "There is an ongoing push to increase [unmanned ground vehicle] autonomy, with a current goal of ‘supervised autonomy,' but with an ultimate goal of full autonomy." Other countries, including China, Germany, Israel, Russia, South Korea, and the United Kingdom, have also devoted attention and money to autonomous weapons.

The fully autonomous sentry robot and aircraft alluded to above are in fact based on real weapons systems. South Korea has deployed the SGR-1 sentry robot along the Demilitarized Zone with North Korea, and the United States is testing the X-47B aircraft, which is designed for combat. Both currently require human oversight, but they are paving the way to full autonomy. Militaries want fully autonomous weapons because they would reduce the need for manpower, which is expensive and increasingly hard to come by. Such weapons would also keep soldiers out of the line of fire and expedite response times. These are understandable objectives, but the cost for civilians would be too great.

Taking action against killer robots is a matter of urgency and humanity. Technology is alluring, and the more countries invest in it, the harder it is to persuade them to surrender it. But technology can also be dangerous. Fully autonomous weapons would lack human judgment and compassion, two of the most important safeguards for civilians in war. To preserve these safeguards, governments should ban fully autonomous weapons nationally and internationally. And they should do so now.

Ralph Orlowski/Getty Images

Democracy Lab

Dreams of Their Fathers

Cambodia's people want freedom. Can Obama deliver?

The question isn't whether President Obama will broach human rights in Cambodia. The question is: Why should he? The answer rests with Cambodian Prime Minister Hun Sen, who hopes to rule until he's 80 or 90 years old, giving him at least another 20 years in power. Effectively, Hun Sen has no opposition, and hasn't for years. His leading rival, Sam Rainsy, lives in self-imposed exile in France. As for other opponents, Hun Sen has pledged to "make them dead." And for demonstrators: "I will beat all those dogs and put them in a cage."

Hun Sen ranks among the world's longest-serving leaders, with more than 27 years on the job. During that time, at least 300 Cambodians have been killed in politically motivated attacks, according to a scathing Human Rights Watch report released last week. The document claims that "[k]illings, torture, illegal land confiscation, and other abuses of power are rife around the country." This year brought more blood: environmental activist Chut Wutty, shot and killed while investigating illegal logging; 14-year-old Heng Chantha, shot and killed among villagers protesting eviction from their land; and environmental journalist Hang Serei Odom, found dead in the trunk of his car with ax wounds to his head.

This is the scene Obama enters as he arrives in Phnom Penh for the Association of Southeast Asian Nations (ASEAN) and East Asia summits Monday and Tuesday. The U.S. visit stems from a diplomatic and economic "pivot toward Asia," which includes a stop in Burma, an updated defense cooperation agreement with Thailand, and, according to the Washington Post, an uptick in counterterrorism assistance to Cambodia (among other countries).

To the Cambodian people, Obama's stop symbolizes the bright, beaming hope of the world's preeminent democracy.

"All Cambodians are focusing on Obama's visit," says Saing Soenthrith, a reporter and editor at the English-language Cambodia Daily, established in 1993. Cambodians hope Obama pressures Hun Sen on human rights, he says. "Our Cambodian people need Obama's help."

But analysts suspect human rights will take a back seat on this ride. The Obama administration has bigger geopolitical fish to fry. "Overall the stakes are much higher: To keep a U.S. foot in the door with Cambodia in an effort to balance Cambodia's growing relationship with China," says Donald Jameson, a former U.S. embassy officer in Phnom Penh during the 1970s who closely tracks the country. Jameson predicts Obama or someone on staff will craft a "general throwaway line" about human rights, "so that the U.S. can credibly argue that it has discussed this with the Cambodian government."

The China factor is big and wide. A territorial uproar in the South China Sea pits China against several ASEAN nations, as well as Japan. The dispute "is getting to be quite aggressive and threatening to many of our traditional allies," says Joel Brinkley, author of Cambodia's Curse: The Modern History of a Troubled Land. "This administration is very intent on increasing and empowering our presence in that part of the world."

China also boasts a massive economic footprint in Cambodia unlikely to be matched by any other nation: a reported $8.2 billion of investments in the past six years, with another $2.5 billion in infrastructure and loans in the works. According to the Center for Strategic & International Studies, China's 2011 investments in Cambodia -- totaling nearly 15 percent of the country's GDP -- exceeded those of the U.S. tenfold. "China has spent billions of dollars trying to buy Cambodia," Brinkley says. That money has helped build an infrastructure and economy unrecognizable from that of just five years ago. Roads, bridges, factories, skyscrapers: Cambodia's pastoral landscape is changing.

Those shifts underscore a crucial point. For all his transgressions, Hun Sen has sown two key seeds while cultivating his power. "He has imposed stability on a country that has experienced more than thirty years of turmoil and civil war," Jameson writes in a March 2011 East-West Center report. And he has opened his country "to virtually unrestricted foreign investment." The Obama administration likes political stability and economic expansion. Both decrease the likelihood of unrest, which would threaten both U.S. and Chinese interests.

The thing is, Chinese money comes with no humanitarian strings attached. And, "foreign leaders coming to Cambodia and lecturing Hun Sen accomplish nothing," Brinkley says. "Nothing."

Human rights issues in Cambodia are "unrelated to visits by foreign potentates," says historian David Chandler, one of the foremost experts on modern Cambodia. Hun Sen has a history of excoriating foreigners who critique him. Recently, when the United Nations special rapporteur for human rights in Cambodia urged Cambodian authorities to improve conditions for next year's national elections, Hun Sen reportedly said he "should not write stupidly about Cambodia."

But Cambodians still have hope for change. Last week, villagers facing eviction from their homes near Phnom Penh's international airport painted their rooftops with "SOS" signs accompanied by pictures of Obama. Eight were arrested. Their message is gone, but the sentiment remains.

And theirs is not the only plea for help. Land-rights activists rallied outside the U.S. Embassy three times in the past week, asking Obama to address forced evictions and land confiscations, according to Long Kimheang, senior communication officer for the Housing Rights Task Force. But the people were "forcibly evicted by hundreds of riot police, military police, secret police, and Phnom Penh security men."

Some U.S. dignitaries are on the activists' side. Twelve members of Congress signed an October 31 letter urging Obama to prod the Cambodian government to clean itself up before the next national elections, set for July 2013.

Human rights are indeed on this week's agenda for discussion. According to a Cambodia Daily interview this week, Samantha Power, senior director for Multinational Affairs and Human Rights on the National Security Council, says she will hold roundtable discussions on a broad menu covering elections, human trafficking, women's rights and other issues.

Some interested parties had hoped for more. "Initially it was all sort of sunshine and light. It looked like labor would be a huge focus," says David Welsh, country program director for the Solidarity Center, an AFL-CIO affiliate. "That, for whatever reason, has been sidetracked... but it created huge expectations."

The momentum comes from Secretary of State Hillary Clinton's visit in July, when she gave a buoyant speech outlining a vision of economic progress bolstered by workers' rights. Child labor, forced labor, and discrimination has to end, Clinton asserted. The right to join a union or collectively bargain is a universal right -- one that, properly exercised, also promotes economic growth, she argued. Standing up for the right to organize is "both right and moral, but it is also smart and strategic." Workers who are paid more buy more goods. And that makes the economy grow.

"For the secretary of state to talk about trade union rights and collective bargaining on that level of specificity was amazing," Welsh says. It showed that geopolitical discussion of business, trade and markets "doesn't have to happen in a vacuum that excludes human rights and proper labor law." Obama could follow suit, emphasizing economy and rights. "You can have both, and I hope he takes a strong line on both."

After all, the United States and Cambodia share market interests. "The largest investors in the country in the garment industry remain American companies like Gap and Levi's and Walmart." American buyers still hold sway, Welsh says, "despite the massive Chinese investment here."

Chinese influence won't end, and Cambodians know that. Chinese traders roamed the region well before the 13th century diplomat Chou Ta-Kuan wrote of his journey from Peking to the ancient Khmer capital of Angkor. The Chinese presence endures. That, Cambodians expect.

But from the United States they want something different: a strong force to defend democracy and condemn the violation of human rights. Obama's visit gives Cambodians new hope, which people need, says Saing Soenthrith of The Cambodia Daily. If Obama does not press human rights issues, he says, Cambodians will live "under darkness of Hun Sen's power."

On some levels, Cambodians have sought and welcomed saviors from the horrors of their recent history, but they've often been let down. In 1979, the Vietnamese ousted the genocidal Khmer Rouge -- and then implemented a repressive decade-long occupation. Khmers still debate whether to call that time liberation or invasion. In 1993, the United Nations brought the country's first-ever democratic elections -- widely seen as flawed. In 1998, amid weeks of violent post-election protests, Cambodians erected a camp called "Democracy Square," with demonstration signs written in English. Protesters told foreign journalists they wanted the United States to set the people free. That didn't happen.

Today, Cambodians look to Barack Obama, president of the world's most powerful democracy, which espouses the very ideals many Cambodian people seek. The president's preceding visit to Burma will show a country progressing in human rights "and Cambodia should do the same," says Hing Channarith, CEO of the Cambodian Children's Advocacy Foundation, an NGO that aids impoverished rural Cambodians. It's a telling statement, considering Burma is just opening up while Cambodia's democracy dates back 20 years.

For Cambodians, the real test of Obama's visit will come next summer, when Hun Sen is, once again, up for reelection. Those polls must be free; Sam Rainsy must have his run, Channarith says. The world must watch and take account.

If not, Channarith says, "then Mr. Obama's visit is nothing."

Photo by JEWEL SAMAD/AFP/Getty Images