The Peril and Promise of Big Data

The White House report on information overload is underwhelming. But it's a start.

The White House released its 79-page report on Big Data on May 1, following a year of scandal and controversy as citizens, activists, and technologists dealt with the revelation that it wasn't just Facebook and Google collecting data on U.S. citizens, but the NSA too.

With shockwaves from Edward Snowden's defection continuing to reverberate across Washington, the broader public increasingly sees the U.S. government as having much too much technical capability for collecting, storing, and processing individual-level data, which in turn allows it unprecedented access into the everyday lives of millions of people -- including U.S. citizens. Meanwhile, other federal employees process retirements with an archaic system where paper forms are filled out by hand, tens of thousands of government computers still run Windows XP (first released before 9/11 and no longer supported by Microsoft), and even the U.S. president's leading policy initiative -- healthcare -- faltered on the shoals of a wrecked website.

There was a time during the Cold War -- and up until the early 1990s -- when the government defined the bleeding edge of innovation -- GPS, voice recognition software, even the Internet itself began in government agencies. But the government has ceded that advantage and it is now largely playing catch-up with the private sector: catching up the IT systems on its workers' desks, catching up on net-neutrality and other Internet regulations, and now catching up on the value -- and hype -- of big data. Concerns about privacy and big data pre-date the Snowden leaks, but the revelations about classified NSA programs have cast a very bright light on just how the U.S. government makes use of big data across its vast departments and agencies.

Here we see an odd and fascinating paradox of American government in the 21st century: incredible technical ability for collecting and analyzing digital information in parts of the intelligence community alongside near complete paralysis to leverage even the most basic commercial technologies to solve more mundane elements of daily governance.

The White House report -- written by U.S. President Barack Obama's Council of Advisors on Science and Technology -- is ultimately an accounting of government responsibilities, but it serves as a useful point of departure for a wider array of conversations regarding privacy, opportunity, and limitations of the new world of ubiquitous sensors and permanent digital data exhaust. It clearly and concisely provides some useful throat clearing on a vast topic, defining terms and framing key parts of the debate; the footnotes alone will be of great use to researchers and commentators. The report, however, is narrow in its scope, focusing on public policy and governmental applications rather than a more sweeping review of the myriad commercial innovations and intrusions most Americans encounter on a daily basis; as a result it is far from definitive -- teeing up more questions than it answers. In fact, nearly all of its policy recommendations are calls for more research or public comment.

So, what then, if anything, can we take away from this multi-month review?

First, there are real perceived limits to the efficiencies of data integration. The tension between the convenience of data integration and fears of government overreach is palpable. We want the government to know enough to be useful and efficient (and clearly we lambast Washington when it isn't), but we remain deeply uncomfortable with widespread data sharing. These are the twin promises and perils of the information revolution and they continue to dominate conversations in D.C., Silicon Valley, and points in between. The White House report itself highlights a case in point: the 2013 shooting at the Naval Yard. The assailant gained access to the base by dint of having active security clearance -- a clearance he retained despite multiple arrests.

Surely the information systems supporting law enforcement and clearance investigations are linked? Alas, no. In a scathing op-ed last year, the Center for Strategic and International Studies' John Hamre decried the arcane (and seemingly ineffectual) clearance process. Calling it "pathetic," he noted "I have dedicated 38 years of my life to America's national security. I know there are spies in our midst. We can improve security and save money simultaneously. But our country needs a system built for the 21st century."

While I for one can personally see the appeal of a system that auto-populates my security clearance paperwork with my home address from the U.S. Postal Service (or Amazon), overseas travel data from Customs and Border Protection, and employer data from LinkedIn (or the IRS), I don't need the Defense Department to know about my unpaid D.C. parking tickets. And while the White House report documents these underlying tensions, it proposes few real remedies.

Second, analytics don't make policy. There is incredible opportunity for innovations in public policy based on advances in data collection and integration. The report highlights health care and public education as obvious entry points, including opportunities for online education in remote areas and new capabilities for researching student learning techniques alongside digitized and personalized health prescriptions. But there is nothing inherent in data -- big, small, or otherwise -- that determines appropriate policy interventions.

Everyone knows "correlation isn't causation" but maybe we need to learn another mantra: "correlation isn't policy." Or as Evgeny Morozov noted in the New York Times last year, "a problem tackled through correlations alone lends itself to a very different set of solutions than a problem mapped out in all its causal complexity." The report's sections on discrimination are instructive here. For example, in Boston "the Street Bump team [developing an app to report road conditions] also identified a potential problem with deploying the app to the public. Because the poor and the elderly are less likely to carry smartphones or download the Street Bump app, its release could have the effect of systematically directing city services to wealthier neighborhoods populated by smartphone owners." The experience of thinking through the likely use-cases, target populations, and resulting data led the team to highlight possible inequalities which would have resulted in skewed policies. Bad data -- no matter how big -- yields flawed policies. A causal story, complete with deep appreciation of the underlying data generating process, is critical for designing policy interventions.  

Further, neither data -- nor data scientists -- can inform the public on how to balance the tradeoffs in these policy choices. Most major policy debates require us to confront questions of equality and opportunity, privacy and security, or individual rights versus societal benefit. We don't just want to protect ourselves from terrorists; we also want free-flowing travel and trade alongside constitutional privacy protections. If crime and education levels are correlated, how should we design policy responses? If the protests and violence in Ukraine had been clearly forecast, should we have intervened? These are inherently political questions about risks and resources, and even the most sophisticated sensors and algorithms cannot adjudicate these discussions. 

This brings us to our final take-away: The information revolution is too important to be left to engineers alone. (It may also too important to be left to Congress, a related but separate issue.) The report surprised many by calling for new investments in "cross-cutting research that involves not only computer science and mathematics, but also social science, communications and legal disciplines." This means more federal research monies to academic social science departments and law schools, and (hopefully) more resources for providing technical training to those outside of traditional engineering schools. (The National Science Foundation's IGERT program serves as a great inter-disciplinary model to build upon, as it provides robust, integrative data and technology training for graduate students in a variety of fields, including social science and public health.) To be clear, these are not problems that lawyers or social scientists or philosophers can resolve alone. The task of designing effective policies that adequately protect civil liberties and prevent discrimination while also fostering innovation is a tall order. But there are two reasons investments in non-engineering fields are necessary.

In my experience, engineers fixate on technical challenges. In general, that's good -- you don't want the engineer building your dam to be distracted by Rawlsian arguments over who should pay for it. But you also wouldn't let the engineer design the broader conservation strategy, lest you be left with water-tight dams and no wetlands or downstream drinking water. The same tensions play out in the defense and intelligence communities. The technical abilities of the NSA are forcefully impressive -- and that's the problem. Furthermore, when innovators push social and political boundaries, technical challenges aren't the only limitations. And yet, those are really the only limits most engineers are trained to care about. Or worse, they see legal restrictions as no different from technical ones merely to be overcome with cleverness and caffeine.

Conversely, those on Capitol Hill and elsewhere in Washington's vast policy apparatus are not trained technologists. In fact, many revel in their ignorance. In the debate over the Stop Online Piracy Act, one congressman instructed the House Judiciary Committee: "Let's bring the nerds in and get this right." The lack of technical savvy amongst staffers and lawyers doesn't just limit their ability to craft meaningful policies, it also limits their ability to challenge the architects and operators of these systems -- it prevents a dialog among equals. In that same hearing, Rep. Mel Watt declared, "As one who acknowledged in his opening statement that he was not a nerd and didn't understand a lot of the technological stuff, I'm not the person to argue about the technology part of this," only to then dismiss the opinions of engineers and network experts who highlighted the risks of SOPA. Conversely, those same lawmakers struggle to properly grill members of the intelligence community when they discuss intercepts and online surveillance.

This White House's new data report is just the beginning of a long-overdue conversation on how we balance the opportunities for better and more efficient governance promised by big data with the intrusions on privacy and risks of discrimination. Its recommendations, while tepid, would constitute meaningful first steps in informing this discussion. But in order to move forward, political and industry leaders are going to have to move past the easy conversations and can-kicking studies to make some difficult decisions about how to even out the government's ability to manage and collect complex data, modernizing the Department of Health and Human Services, the IRS, and the Department of Veterans Affairs, while possibly constraining the NSA. And ultimately we'll have to engage in the painful work to build a consensus on how much efficiency we're willing to forgo in the name of privacy, and vice versa.



No, Sanctions Didn't Force Iran to Make a Deal

Reaching out to Tehran, not smashing its economy, is what made the nuke deal possible -- and it could have happened much earlier.

In what is perhaps the central myth of the Cold War, President John F. Kennedy is said to have stared down Nikita Khrushchev during the Cuban Missile Crisis and refused to give an inch, forcing the Soviet premier to capitulate to his steely will and America's superior military might. As Leslie Gelb, president emeritus of the Council on Foreign Relations, put it in Foreign Policy, "Mythologically, Khrushchev gave everything, and Kennedy gave nothing." This false standard, according to Gelb, became the gold standard for American statecraft going forward: Never compromise, just stare down your enemies and force them to capitulate.

In reality, of course, Kennedy did compromise. Only by quietly withdrawing its Jupiter missiles from Turkey did the United States avoid a nuclear confrontation with the Soviet Union. But for several decades, the Kennedy administration managed to keep this essential part of the story a secret. By the time the compromise was revealed 16 years later, in a book by historian Arthur Schlesinger, the myth had grown so strong that the truth could not unseat it.

Today, another, equally destructive myth is being forged.

That myth -- promoted by officials in President Barack Obama's administration as well as powerful lawmakers like Sen. Bob Menendez (D-N.J.) -- is that crippling sanctions brought the Iranian regime to its knees, forcing it to rush to the negotiating table to beg for mercy. In this narrative, the breakthrough in nuclear talks is credited to the Obama administration's unprecedented economic pressure, which has essentially locked Iran out of the international financial system. And like JFK before him, Obama did not compromise with Iran. The mythical gold standard was met.

Except it wasn't.

Sanctions are neither the reason for the breakthrough, nor the impetus behind the government of Iranian President Hassan Rouhani's openness to talks. They also did not get Rouhani elected.

The idea that the United States has the ability to engineer the outcome of elections in a country that is thousands of miles away, with which it has no trade, where it has had no diplomatic presence for 35 years, and where only a handful of current U.S. diplomats have ever served or even visited, expands the concept of arrogance to new and exciting frontiers.

In reality, last year's elections were a continuation of the fraudulent 2009 elections -- some might argue, the completion of that tense chapter. Iranians wanted change in 2013, just as they did in 2009 -- before the imposition of Obama's sanctions. The last four years under President Mahmoud Ahmadinejad had been worse than the first four. Repression had intensified, the security atmosphere in Iran made the heyday of McCarthyism look like the enlightenment, corruption and economic mismanagement was at an all-time high, and the hardliners had criminalized everything from academia to tourism. The population was suffocating. The regime had thwarted Iranians' vote for change in 2009, and few believed they would even bother to cast their votes in 2013.

This was the critical question -- voter turnout -- because hardliners in Iran only tend to win elections under two circumstances: When they cheat or when they convince the population that they will cheat. In the latter case, they suppress voter turnout and enable a core group of supporters of the regime to swing the outcome of the election.

In the end, a range of forces enabled Rouhani and his political allies to convince a large portion of the electorate that hardliners simply could not repeat the charade of 2009. Reformists led by former President Mohammad Khatami and political centrists supporting former President Hashemi Rafsanjani formed an unprecedented coalition in support of Rouhani, while conservatives failed to coalesce around a single candidate of their own. The wounds of 2009 were still open, meanwhile, and internal rifts between the ruling elite suggested that the regime could not survive the delegitimizing effects of another election scandal. As a result, Rouhani could convincingly tell the crowds at numerous campaign stops that, "2013 will not be like 2009."

As election day approached, Rouhani surged in the polls and to the surprise of many -- perhaps even his team -- rolled to the presidency: With 72.7 percent turnout, Rouhani won a landslide, first-round victory with 50.7 percent of the vote.

This outcome was also determined by a bit of luck. A poll conducted by Tehran University and the University of Maryland immediately after the election revealed that strategic voting by supporters of Rouhani's rival, Tehran Mayor Mohammad Bagher Ghalibaf, pushed Rouhani over the 50 percent mark. Since the elections were expected to go to a runoff and many Ghalibaf supporters believed he was a shoe-in for the second round, they instead cast their first-round ballots for Rouhani -- their second choice -- to secure a Rouhani-Ghalibaf runoff (the poll revealed that 24 percent of Rouhani voters actually preferred Ghalibaf.) But the Ghalibaf supporters overdid it. Thanks in part to their strategic voting, Rouhani managed to reach just above the 50 percent threshold, eliminating the need for a runoff. 

The Tehran University/University of Maryland poll also directly refutes the idea that sanctions got Rouhani elected: Only 2 percent of Rouhani's supporters listed the lifting of sanctions as a reason for supporting him. Twice as many -- 4 percent -- voted for him because he was a clergyman. Seven percent cited his ability to fix the economy. A later poll by Zogby International revealed that three out of the five most important issues to the Iranian electorate pertained to civil liberties, while a whopping 96 percent reported that sanctions were worth it in order to retain the country's enrichment right.

Rather than crediting sanctions for this unexpected outcome -- without a shred of evidence -- it should be acknowledged that Iran's presidential election was unpredictable. The election could have easily produced a different result: What if Rouhani had surged earlier in the polls, reducing the degree of strategic voting among Ghalibaf supporters? What if Khatami had failed to convince the other reformist candidate, Mohammad Reza Aref, to withdraw his candidacy and throw his support behind Rouhani? What if the conservatives had succeeded in uniting around one candidate? And most importantly, what if a greater segment of reform-minded Iranian voters had decided not to vote?

* * *

Equally questionable is the argument that sanctions brought Iran to the negotiating table -- or even that they are the driving force behind Rouhani's appetite for diplomacy. Such claims ignore the fact that the team around Rouhani has had a long history of pursuing a more conciliatory policy towards the West, including on the nuclear issue.

Rouhani headed Iran's Supreme National Security Council, the equivalent of the U.S.'s National Security Council, in 2001, when Tehran helped Washington topple the Taliban in Afghanistan. According James Dobbins, who served as President George W. Bush's envoy to Afghanistan in the months after the 9/11 attacks, Iran provided crucial intelligence as well as military and political support to the United States -- long before any of the current sanctions were imposed. Later, it was Rouhani's current foreign minister, Javad Zarif, who coordinated with Dobbins to secure support for the new post-Taliban constitution in Afghanistan. The Iranians hoped that their assistance in Afghanistan would open a new chapter in U.S.-Iran relations, but Bush was not interested. Instead, he included Iran in his "axis of evil," effectively killing the collaboration in Afghanistan.

But the reform-minded team in Iran did not relent. As I describe in my 2007 book, Treacherous Alliance: the Secret Dealings of Israel, Iran and the U.S., Tehran submitted a comprehensive negotiation proposal -- a grand bargain -- to the Bush administration in 2003. Zarif was one of the authors of that document. Among other things, Iran offered to make its nuclear program fully transparent (at the time, it had only 164 centrifuges, compared to the 19,000 it has now), disarm the Lebanese militant group Hezbollah, and indirectly recognize Israel. But once again, the Bush White House rejected Iran's outreach.

Two years later, during his last months as head of Iran's nuclear negotiating team, Rouhani made one final attempt to meet the West halfway. In March 2005, he instructed Zarif -- then Iran's U.N. ambassador -- to submit to the Europeans a proposal that would have limited the number of Iranian centrifuges at 3,000. Iran negotiated directly with the EU at the time because the Bush administration refused to come to the negotiating table. But the Europeans never responded to this offer, mainly because they knew Washington would reject any deal that allowed for even one spinning centrifuge on Iranian soil. A senior official in the Obama administration told me a few months ago that the United States would jump at such a proposal today because of the significant progress the Iranian nuclear program has made since 2005 -- sanctions notwithstanding.

The fact that the pragmatic faction within the Iranian government has on numerous occasions offered more attractive nuclear proposals to the West -- prior to the crippling economic sanctions imposed by Obama -- fundamentally undermines the notion that sanctions were needed to reach a deal.

While it is true in a limited sense that sanctions provided the United States with added leverage (assuming it can lift them as part of a deal), the other side of the equation is all too often conveniently forgotten: During this same period, Iran aggressively expanded its nuclear capabilities, which in turn provided it with added leverage over the West. Iran increased its centrifuge count from 3,000 to 19,000 and built a number of advanced centrifuges it didn't have back in 2005. It also amassed thousands of kilograms of low enriched uranium, as well as roughly 200 kg of 20 percent enriched uranium -- of which it had none prior to 2010.

If sanctions gave the United States more leverage, then it's also true that 19,000 centrifuges gave Iran additional leverage. Indeed, some in Tehran contend that Iranian centrifuges forced the United States to the table. After all, it was the United States that for years refused to engage in negotiations with the group of world powers known as the P5+1 -- not Iran.

At a minimum, the growth of Iranian capabilities limited America's options. As one White House official who is involved in the negotiations told me, "We are negotiating because the Iranians are on the cusp of a fait accompli."

* * *

In reality, it was neither the sanctions nor Iran's centrifuges that produced the current breakthrough. The diplomatic opening came about for the same reason it did during the Cuban Missile crises: Both sides compromised. Tehran stopped advancing sensitive parts of its program and agreed to greater transparency. And Washington finally accepted enrichment on Iranian soil in the November 2013 interim agreement. Tehran had long insisted that if its enrichment was accepted, it would agree to transparency as well as restrictions.

For all practical purposes, accepting Iranian enrichment is the modern equivalent of removing Jupiter missiles from Turkey. If this unrealistic and legally questionable red line had been discarded earlier, the breakthrough could have been achieved much earlier -- long before the Obama sanctions were imposed.

Mohamad ElBaradei, the former head of the International Atomic Energy Agency, has written that the "interim agreement, facilitated by Rouhani's low-key diplomacy, could have been reached 10 years ago." But, he added, it took the "West a decade to realize that bare-knuckle competition for regional influence was not a viable strategy for dealing with Iran."

Obama missed one such opportunity for compromise in May 2010, when Brazil and Turkey convinced Iran to accept an American proposal to ship out 1,200 kg of its stockpile of low enriched uranium (LEU) in exchange for fuel pads for its Tehran Research Reactor. Despite the fact that Obama welcomed Turkey and Brazil's efforts and spelled out the specific conditions Iran needed to accept in a letter dated April 20, 2010 -- all of which Tehran accepted -- the United States reneged on its promise and rejected its own proposal.

There were numerous reasons Obama chose to reject the Tehran Declaration, as the Turkish-Brazilian deal came to be called, but perhaps the two most important ones were the unstoppable momentum of sanctions and the issue of Iranian enrichment.

After the failure of the 2009 nuclear negotiations, support for sanctions on Capitol Hill was strong and growing. The Obama administration chose not to oppose this pressure, but rather to delay until it had first secured a U.N. Security Council resolution imposing sanctions. Unbeknownst to Brazil and Turkey, Russia and China approved a sanctions resolution one day before the negotiations with Tehran were due to begin. Once the deal was reached, Obama had to choose between a diplomatic breakthrough and sanctions.

He chose sanctions.

Obama feared that even if he accepted the Tehran Declaration, Congress would still impose sanctions. That would have complicated matters and broken the unity Obama had carefully forged in the U.N. Security Council against Iran. A senior Obama official told me that Congress was coming at the administration "like a steam roller" and that Obama simply lacked the political space to take "yes" for an answer from Iran.

But sanctions were never necessary to get Iran to agree to a deal. Rather, sanctions were needed to pacify domestic political forces in the United States and to give Obama the space he needed to pursue diplomacy down the road. Sanctions were in this regard a domestic-policy tool, not a foreign-policy tool.

The second reason Obama chose to reject the Tehran Declaration was that it explicitly recognized Iran's right to enrich. Even though accepting the declaration would not have made the United States a party to it, the White House feared it would violate America's long-standing "red line" on enrichment in Iran -- the same red line Obama wisely shelved in 2013 in order to get a deal with Tehran.

Yet the myth that sanctions produced the current diplomatic breakthrough persists. Lawmakers continue to argue for more sanctions, even though such action would cause the talks to collapse, claiming that since sanctions brought Iran to the table, more sanctions will give the United States even more leverage.

If the myth of the sanctions success prevails, American foreign policy will be led down a perilous path. A false and dangerous blueprint for dealing with proliferators and international disputes in general will emerge: Forget diplomacy, never compromise, impose sanctions, threaten war -- and hope for the best.

With Iran, thanks to the quiet compromise on enrichment, war is more distant than ever since the crises erupted. The world may not be as lucky next time it goes down an all-out sanctions path.

Getty Images