Why the World Can't Have a Nate Silver

The quants are riding high after Team Data crushed Team Gut in the U.S. election forecasts. But predicting the Electoral College vote is child's play next to some of these hard targets.

After a presidential election that Nate Silver and a smattering of other statistical modelers forecast with remarkable accuracy, quantitative enthusiasts -- quants -- are talking some hard-earned smack. "This is about the triumph of machines and software over gut instinct," Dan Lyons extolled at the tech blog ReadWrite. "The age of voodoo is over. The era of talking about something as a 'dark art' is done. In a world with big computers and big data, there are no dark arts."

If only. As a practicing forecaster who prefers algorithms to expert judgment, I'm thrilled to see statistical forecasting so publicly vindicated, but I'd also like to engage in a bit of expectations management about how quickly these methods might transform international politics. As sci-fi writer William Gibson famously said, "The future is already here -- it's just not very evenly distributed." As imperfect as they still are, statistical forecasts of U.S. elections are on the leading edge of that distribution. Meanwhile, most things foreign-policymakers care about are closer to the far edge.

To see why, it's important to understand that Silver and his ilk didn't succeed simply by using "math" instead of "gut." Yes, the method matters, but statistics isn't alchemy. To build forecasting models that work well, you need reliable measures of things that are usefully predictive. Even tougher is that you need those measures not just for today, but also for a long- and broad-enough swath of history to be able to test your beliefs about what predicts what against some hard evidence before diving into prognostication.

Routine elections in rich countries like the United States are some of the softest targets in political forecasting. Rules are transparent; high-quality data, including surveys of would-be voters, are often available; and the connection between those data and the outcome of interest is fairly straightforward.

Even in these relatively easy cases, though, forecasting can still be challenging. In 2010, Silver -- the man the Economist called "the finest soothsayer this side of Nostradamus" -- tried to predict the outcome of parliamentary elections in Britain and missed pretty badly.

Of course, elections in obviously authoritarian regimes are even easier to forecast. Until Mikhail Gorbachev rolled around, no one needed a model to predict who was going to win election to the Supreme Soviet of the USSR. The task is much tougher in competitive authoritarian regimes, where subtler forms of coercion tilt the field in favor of one party, but don't quite guarantee a specific outcome.

Take October's legislative election in Georgia, where the Georgian Dream coalition upset President Mikheil Saakashvili's ruling United National Movement after late opinion polls appeared to show a solid lead for the incumbents. As Mark Mullen, the chairman of Transparency International Georgia, pointed out, what simple readings of those pre-election polls overlooked was the large share of respondents -- a whopping 46 percent -- who refused to pick a favorite. According to Mullen, that refusal was probably driven by fear of "taking risks that could have put [respondents] on the wrong side of the authorities." In an atmosphere of fraud or intimidation, it is a lot harder to make accurate forecasts, even in the rare cases for which we have professional polling data.

When it comes to predicting major political crises like wars, coups, and popular uprisings, there are many plausible predictors for which we don't have any data at all, and much of what we do have is too sparse or too noisy to incorporate into carefully designed forecasting models. In a perfect world, forecasters would routinely receive survey data that would shed light on the sentiments and intentions of the people who might engage in these activities. In the real world, it's tough to get honest answers to questions about people's willingness to participate in extralegal activities like protests or rebellion -- and that's assuming they could even be reached in the first place.

Absent direct measures of interests and intentions, we're forced to rely on measures of structural conditions that might shape political behavior. This is what some forecasters of presidential elections do, using things like incumbency, job growth, and changes in income to generate predictions months ahead of the vote. These kinds of models perform pretty well, but the forecasts they produce are typically less accurate than their poll-averaging counterparts.

The same logic holds in international affairs. Pretty much every theory of domestic political instability starts from the assumption that, other things being equal, poorer countries are more susceptible to crisis than wealthier ones. Simple, right? Just toss per capita GDP in your algorithm and move on to the next predictor.

Not so fast. As it happens, GDP estimates are produced by government agencies whose data-making capacity is directly related to the thing they're trying to measure. Some countries, including Cuba and North Korea, don't even report national economic statistics to the international bodies that collect them. And that's close to the best-case scenario. Reliable measures of many other oft-mentioned risk factors, like unemployment and income inequality, were simply unavailable for almost all countries until very recently, and coverage is still largely confined to richer parts of the world.

These gaping holes in the historical record don't make it impossible to generate useful statistical forecasts of international affairs. They do mean, however, that the forecasts we can make are much less accurate than the ones the poll-averaging modelers can produce for U.S. elections.

For rare events like coups or outbreaks of civil war -- in most years, only a few of these events will occur worldwide -- it's easy to be right almost all the time by saying nothing will happen anywhere, but that's also not particularly useful. The harder task is identifying where and when the occasional exceptions will occur without crying wolf too often.

This problem bears some resemblance to forecasting U.S. presidential elections, in which most of the 50 states dependably vote Democrat or Republican; the hard part is predicting the dozen or so swing states. In international politics, there are many cases that seem reliably "immune" to certain crises, and there's often also a small but self-evident set of usual suspects. It's the small but critical set of cases in between those two extremes that make us work to earn our paychecks.

Again, though, difficult does not mean impossible. As Pennsylvania State University political scientist Philip Schrodt has pointed out, well-designed models have achieved a respectable level of accuracy on a range of forecasting problems, including outbreaks of civil war and mass atrocities and the occurrence of coups d'état. Still, these models usually aren't as precise as we'd like. For every high-risk case that suffers a crisis, there is usually at least a handful of them that don't, and occasionally a supposedly low-risk case just plain surprises us.

Data gleaned from the deluge of information now pouring over the Internet may soon help fill some of these gaps, but we're not there yet. In the meantime, we must create forecasts with the data we have, not the data we want. It's great that statistical forecasters won wider respect for their methods by nailing the outcome of this year's U.S. presidential election. But it's important for people to appreciate that not every forecasting problem can be solved by sprinkling it with math and silicon.



Relationship Advice

Don't believe the loose talk of Barack Obama exacting vengeance on Benjamin Netanyahu -- the two leaders need each other too much to let old grudges get in the way.

In the wake of U.S. President Barack Obama's reelection victory, some have suggested that he will pursue a feud against Israeli Prime Minister Benjamin Netanyahu, given the two leaders' disagreements over how to pursue peace with the Palestinians and deal with the Iranian nuclear threat.

There is no denying that the relationship between the two leaders has been rocky. Yes, Obama believed Netanyahu had wrongly lectured him about borders in front of the media in the Oval Office in May 2011. Netanyahu has his own grievances: He was upset that he could not get a September meeting with the president to discuss Iran, for instance. As Israeli Interior Minister Eli Yishai put it the day after the U.S. election, "It seems like it is not such a good morning for Prime Minister Benjamin Netanyahu."

While it is safe to assume Netanyahu personally preferred Mitt Romney, he in fact did not endorse Obama's Republican challenger -- despite every Israeli reporter's efforts to entice him into doing so. As one Netanyahu aide put it privately, when it comes to the U.S. election, "our room to maneuver on this issue has the width of dental floss."

But at the end of the day, a settling of scores between Obama and Netanyahu is unlikely. It is counterproductive for the two leaders to focus their energies on the past when they are confronted by an array of challenges that will require them to work together.

Obama's cerebral style toward foreign leaders made Israelis skeptical of him, in part because it was a departure from the bear-hug style of President Bill Clinton. But it is also one reason the United States and Israel will now avoid a public feud. As Dan Shapiro, Obama's former top White House aide and current U.S. ambassador to Israel, told a panel in Tel Aviv on Nov. 7, "The president is a strategic thinker; his policies are not governed by emotion." He termed talks of Obama taking revenge against Netanyahu "ridiculous."

Too much is at stake for both countries to let old grudges dictate policy. It is no secret that the Obama administration views a new diplomatic initiative toward Tehran as integral to its sanctions policy. The potent international sanctions currently in place, combined with diplomacy, are the world's one hope of solving the Iran nuclear crisis peacefully. Nobody can guarantee that Iran will back off from its program, but a U.S.-led offer is still inevitable to test that proposition.

And Israel knows this. Contrary to perception, Netanyahu would also like to see a peaceful end to the crisis -- there is no Gen. Curtis LeMay figure in the Israeli government out to firebomb Iran. Whether it is in the format of bilateral U.S.-Iran talks or the P5+1 (the five permanent members of the U.N. Security Council plus Germany), the United States will want Israel on board with U.S. diplomatic efforts to avoid the prospect of an Israeli strike. This does not give Netanyahu any kind of veto over the U.S. offer to Iran, but it is hard to imagine that the United States would not welcome Israel's thoughts to ensure that the two countries do not act at cross-purposes.

This process will test the personal relationship between Obama and Netanyahu like no other foreign-policy issue -- and it will leave no time for petty score settling. Israel and the United States are going to need to be in closer consultation than ever about Obama's highest foreign-policy priority: namely, ensuring that Iran does not go nuclear and that a nuclear arms race does not break out in the region. Failure would mean the end of Obama's plans to promote nuclear nonproliferation, and it would also cripple U.S. credibility after three administrations -- Democrat and Republican alike -- have vowed that Iran will not get the bomb.

Moreover, those who are playing up the possibility of an Obama-Netanyahu feud believe that the laws of political gravity are suspended for a second-term president. In fact, while reelection can revitalize a president's mandate, political capital remains a finite commodity -- even for a second-term U.S. president. Just ask President George W. Bush, who saw his clout diminished in his second term by Hurricane Katrina, a failed attempt to privatize Social Security, and a debilitating war in Iraq. Bush allocated all his efforts to the 2007 Iraq surge, and administration officials at the time said not much political capital was left over for anything else. As it is, Obama is facing a divided Congress that he will need to win over for a grand budget deal in 2013, which will be central to resolving the country's economic crisis.

This is not an argument for Obama to avoid the Palestinian issue and focus solely on Iran. To the contrary, advancing a two-state solution is essential if Israel is to remain a Jewish, democratic state and not a de facto binational state, and for the Palestinian national movement to have a real future. Moreover, whatever the arguments between the parties at the U.N. General Assembly this month, when the Palestinian Authority is expected to pursue its bid for statehood, it is important that the Palestinian Authority does not go bankrupt. The collapse of the one existing institution that could potentially serve as the core of a new Palestinian state would not benefit the United States, Israel, or the Palestinians.

Once again, Obama is going to need Netanyahu's help to advance these goals -- and the president should assume that the Israeli prime minister will remain on the political scene for some time yet. Polls show Netanyahu is likely to win the Israeli election on Jan. 22, given his merger with the right-wing Yisrael Beiteinu party and his natural alliance with religious parties. Former Prime Minister Ehud Olmert may enter the race looking to unite the center, and he may hint in his campaign about his proven willingness to make tough decisions -- notably, a still officially secret strike against a Syrian nuclear reactor in 2007 and his willingness to make difficult concessions on peace to the Palestinians. The odds favor Netanyahu at this time, however.

An Obama swipe at Netanyahu would not necessarily hurt the current prime minister, but could boomerang against the president. Obama does not have Clinton's endless reservoir of support among the Israeli public, regardless of the important strides in bilateral security between the two countries over the last four years. One poll showed a majority of Israelis favoring Romney.

At the same time, Netanyahu can learn from the past as well. He is shrewd enough to realize that a broad-based unity government is essential if he wants to avoid the disproportionate influence of hard-line elements within his coalition, which could lead him to focus excessively on settlements. With the challenge of Iran and the Palestinian issue, a broad-based government will minimize the prospects of friction with Washington.

Of course, the Palestinians must also do their part for negotiations to be a success. They need to return to the table after leaving in September 2010 over demands for an extended settlement freeze in the West Bank. Obama has every right to insist that both Palestinian President Mahmoud Abbas and Netanyahu provide a sense to the United States of not just what they are willing to gain, but what they are willing to concede if they want the United States to devote precious resources to a renewed push on the Israeli-Palestinian front.

These discussions won't be easy, and they will require close coordination between Obama and Netanyahu to achieve the leaders' mutual goals. Like it or not, the two leaders will look to manage their differences and find a way to work together better than they have in the past -- if only because they must.