National Security

Nate Silver vs. Kim Jong Un

Why we can't figure out what North Korea will do next.

The presidential election is finally over and Nate Silver won. It was a "revenge of the nerds" moment: The data geek bested many of the cool kid pundits and their time-honored tradition of predicting election results based on gut feel, a favorite poll, and a few conversations with unnamed campaign aides and undecided voters in Ohio diners. Silver's 538 model proved what we should have known for years: good election forecasting is all about math, not anecdotes. It's the analysis of big data that produces big insight.

If only improving analysis of American national security threats were that easy. But of course it's not. Because assessing foreign policy challenges is less about crunching numbers and more about understanding the touchy-feely world of intentions -- how allies and adversaries think, what they want, how they feel. Intentions are still the tough stuff of most of our hardest security challenges -- whether China will become a responsible stakeholder or a revisionist power, whether Iran will cross the nuclear redline, whether North Korea will come in from the cold.

North Korea's twenty-something leader, Kim Jong Un, is the poster child of this intentions problem. Even though the DPRK is one of the most closed societies on earth, we know an awful lot about Kim's military. We can spot his country on a map, identify his Navy ships at sea, recognize his army's uniforms across the Demilitarized Zone, and even watch video of the Supreme Leader visiting amusement parks and kindergartens. We also have a pretty darn good idea of his nation's military capabilities, including the approximate number and types of nuclear weapons. Using commercial satellites, Google Earth, and other open sources, my colleague Sig Hecker, along with Frank Pabian, have even pinpointed the likely location of North Korea's third underground nuclear test tunnel, which they believe could be used with two weeks' notice.

The one thing that intelligence analysts inside the government and academic experts outside the government don't have a clue about is what this inexperienced leader with the dour haircut and Mao suit intends to do next. Information about Kim was so tightly controlled that as late as 2009, CIA officials were compiling their dossier of the heir apparent based on secondhand reports from a sushi chef. Given North Korea's penchant to play the reckless rogue, its growing nuclear arsenal, and its close proximity to 28,500 U.S. troops stationed in South Korea, getting a better read on Kim Jong Un's policy preferences and personality is no small matter.

So what does Kim have in mind? Experts are all over the map. Some see Kim's recent moves -- including public statements about ending "belt tightening" and rumored plans to start decollectivizing farms -- as a sign that North Korea might be moving toward Chinese-style economic reforms. A few, like Jay Ulfelder, think that political liberalization may not be so far-fetched. Victor Cha is far more skeptical, warning that Kim is no reformer, despite his love of pro basketball and pizza. About the only points of agreement among Korea watchers are that Kim likes Disney characters and that he's got a nose for throwback fashion styles. But even on this last point, it's still unclear whether Kim's dress is for show or for real -- is it a deliberate ploy to make himself look like his popular grandfather? Or is that just the Hermit Kingdom strongman uniform?

In 2002, Defense Secretary Don Rumsfeld got sneers and jeers for rambling about "known unknowns." But Rumsfeld was onto something. He was trying to make a crucial distinction originally developed by Sherman Kent, the legendary leader of the CIA's analytic branch. Kent argued that there are three types of intelligence:

1. Information that is knowable in the world and known by our government;
2. Information that is knowable in the world but not known by our government;
3. Information that is not knowable to anyone at all.

The first, "knowable and known" category includes things like weapons that can be counted in theory and have been counted by U.S. officials in practice. The number of Chinese aircraft carriers, for example, is well known (they currently have one, a refurbished Ukrainian ship). The second category of intelligence includes things that can be discovered in theory by our intelligence agencies but have not been discovered in practice. We know about the existence of China's carrier, but unless you have been on board or have a spy who has, it's hard to know how the newly retrofitted ship operates under various weather and battle conditions. Carrier performance is known by some in the Chinese navy but probably not by us. The third intelligence category is the doozy: Things that are not knowable to anyone at all. This is the world of intentions.

Assessing even our own intentions turns out to be harder than you might think. Imagine that I asked you to write on a piece of paper where you intend to spend your summer vacation next year. Eleven months from now, you retrieve your piece of paper and compare what you predicted to what you actually did. There is a pretty good chance that your prediction turns out to be wrong, for a gazillion possible reasons. Maybe you took up snorkeling, gave up smoking, bought a house, lost a job, had a baby. Or maybe the change in plan had nothing to do with you -- the hotel closed for renovation, the travel deal expired, the security situation there deteriorated. Life has a way of intervening, making even the best laid plans go awry.

This is the easiest case: It's you predicting your own behavior. Now try predicting the behavior of someone else. Add volumes of information, some of it correct, some incomplete, most of it probably wrong. Extend the time horizon. And then consider that this person is actively trying to deceive you.

Welcome to the world of intelligence analysis.

This is why intelligence agencies have such a hard time understanding whether China's rise will be peaceful, whether Iran is going nuclear, whether Burma's political opening will continue, whether the Muslim Brotherhood is serious about democratization, or what in the world Kim Jong Un will do next. Even Kim may not know the answer to that one. But maybe Joe Scarborough and Nate Silver can bet on it.

Ed Jones/AFP/GettyImages

National Security

Interrogation Techniques

What Bob Schieffer can learn from the CIA.

So here's my bet: The next president will spend a great deal of time worrying about foreign policy issues that Bob Schieffer never mentioned and voters never considered in Monday night's foreign policy debate. And no, it's not just because shit happens. It's actually much worse than that. Looking at the past three elections, I found that presidential debate moderators did a surprisingly bad job of picking the foreign-policy issues that presidents later confronted in office. And here's the really interesting part: the U.S. intelligence community did much, much better, pinpointing serious threats in their annual threat assessments.

In the 2000 election, terrorism was completely absent from all three of the debates between Al Gore and George W. Bush. Sure, it's tempting to put this one into the "who could have possibly known" category. That is exactly what Mitt Romney did Monday night, using this little tidbit of debate history to argue that strong militaries are necessary to defend against the unexpected.

But people did know about the growing terrorist threat before the 2000 debates and the 9/11 attacks. Not just any people. Senior people in the CIA and the FBI.

The CIA warned in both its 1999 and 2000 unclassified annual threat assessments to Congress that terrorism ranked second on the list of threats to U.S. national security. That's right. Second. Just behind the proliferation of weapons of mass destruction. In his 1999 testimony before a public hearing of the Senate Armed Services Committee, CIA Director George Tenet remarked, "there is not the slightest doubt that Osama Bin Laden, his worldwide allies, and his sympathizers are planning further attacks against us. Despite progress against his networks, Bin Laden's organization has contacts virtually worldwide, including in the United States -- and he has stated unequivocally, Mr. Chairman, that all Americans are targets." In March of 2000, six months before the first Bush-Gore presidential debate, Tenet reiterated his concerns, telling Congress again in open session that bin Laden "wants to strike further blows against America" and that the CIA believed "he could still strike without additional warning."

Tenet was not alone. The FBI declared counterterrorism its number one priority in its 1998 strategic plan, three years before 9/11 and two years before the presidential debates. But the media did not notice. The Commission on Presidential Debates did not notice. Moderator Jim Lehrer did not notice. Neither did the two presidential campaigns.

The 2004 foreign policy debates between John Kerry and George W. Bush focused, understandably, on terrorism, homeland security, and the wars in Iraq and Afghanistan. But the candidates also spent time responding to questions about Canadian drug imports and the draft. What was left out? A little country called China. Or more specifically, China's breakneck economic and military rise and its increasing assertiveness in the Asia-Pacific region. Here, too, the CIA was on it, flagging China's rise in its 2004 annual threat assessment several months before the debates began. The question of how to handle China's emergence as a rising power went on to become a major focus of President Bush's second term, it generated Obama's much ballyhooed "Asia pivot" earlier this year, and it remains one of the top foreign policy challenges of the foreseeable future.

The 2004 debates also never mentioned what turned out to be the mother of all foreign and domestic policy issues at the end of the Bush administration: The health of the global economy. To be fair, the global financial meltdown came as more of a surprise to economists and bankers than 9/11 did to intelligence officials. Whether the economic crisis should have been a surprise is another matter -- and a reminder that economists are world class at two things: exuding confidence even when wrong and predicting the past. But I digress.

The big debate miss in 2008 was cyber security. Again, intelligence officials were sounding the alarm but the debate ignored it. The 2008 threat assessment from the CIA director's successor, the director of national intelligence, noted that the threat from cyber attackers -- which included states like Russia and China, non-state organizations like criminal syndicates and terrorist groups, and lone Cheeto-eating hackers -- was large, serious, and growing fast. Yet in the three 2008 debates, John McCain and Barack Obama were never asked what they would do to protect America's military from cyber espionage or disruption; how they would defend America's critical infrastructure like dams, financial systems, and power grids from cyber attack; or how they would work with the private sector to stop billions of dollars of intellectual property theft that many fear could cripple America's competitive advantage in the global economy. Since those debates, the Obama administration has been seized with cyber concerns, creating a new Pentagon Cyber Command and feverishly trying to figure out who should do what in the cyber domain. This month, Defense Secretary Leon Panetta darkly warned that the United States now faces the very real prospect of a "cyber-Pearl Harbor."

Cyber security made a cameo appearance on Monday, but only because Obama mentioned it in passing. Schieffer never asked a cyber question even though cyber security threats have vaulted to number three on this year's intelligence threat assessment. Global climate change got no mention on the debate stage at all, despite growing concerns that rising temperatures could be one of the world's worst threat multipliers, causing water and food shortages and extreme weather events that could create humanitarian disasters and turn fragile states into failed ones and perhaps terrorist havens.

What's gone wrong? Why are presidential debate moderators so bad at picking issues compared to the CIA and the DNI?

Two reasons. First, presidential debates are creatures of the news media, and the news media is obsessed with the news du jour. These are people trained to get scoops, write headlines, cover "the presidential horse race," and link everything they can to today's "pegs." In the media world, faster is better and current is king. But in the real world, foreign policy threats and opportunities take time. They gather. They simmer. They bubble up and die down, reacting to the reactions of others. In the news, the devil usually lies in the details. In foreign policy, the devil often lies in the trend, understanding what looms over the horizon and how it could affect vital national interests tomorrow. The annual intelligence threat assessment is all about horizon-gazing, assessing how current events will play out over time. Presidential debate moderators are all about the here and now, how last night's comments are playing in Ohio this morning. And that makes all the difference.

The second reason presidential debates are poor predictors of future foreign policy challenges is that our election system rewards short-term thinking. Quick wins play well with voters; longer-term strategies to fight emerging threats do not. Presidential candidates are smart people. They respond to incentives. That's why we hear a lot about what they will do "on day one" and precious little about what they hope to put in place for 2050.

It does not have to be this way. In today's presidential election system, candidates increasingly control what they say and how they say it -- running scripted ads, attending scripted events, and reciting scripted lines. But debates are different. They are golden opportunities to press candidates to think and react, to get in front of issues rather than just staying on top of them. To work well, however, the entire debate system is in dire need of an overhaul. A good start would be for moderators to check the headlines less and the intelligence threat assessments more.

CHARLES DHARAPAK/AFP/Getty Images