Midfield General

Why Moneyball Doesn't Work at the World Cup

Forget big data -- measuring performance during this tournament is a losing battle.

Sports analytics -- or moneyball, if you prefer -- is hot right now, and nowhere more so than in football. Something of a latecomer to the gospel of data, the beautiful game is now embracing it with the zeal of the converted. With its fortuitous timing, the World Cup seems like an obvious showcase for the best that football's data experts have to offer. Except it's not.

Football data is big business. Several companies already collect data from on-field events to sell to clubs and, increasingly, fans and media. Opta, Prozone, and Infostrada have been working furiously throughout the World Cup, publishing match data in visually attractive ways to raise their profiles and attract new customers. And there's nothing wrong with that -- these companies have no choice but to take advantage of the tournament's massive global popularity to move product and get wider exposure for their work.

The problem is, as far as statistical analysis goes, the World Cup itself is probably the least friendly event on the football calendar. That's because in sports analytics, most statisticians prefer to work with a large sample of games between teams with generally stable rosters that play each other in a balanced schedule. The bigger and more consistent the sample size, the better chance variations in performance within single games will smooth itself out and allow reliable signals to emerge. That's why a data-rich domestic competition like the English Premier League tends to be the preferred object of analytical study. 

By contrast, the World Cup is a month-long knockout competition. Half of the 32 teams will play a maximum three matches, and only four teams will play the maximum of seven. Their opponents depend on the initial draw and the final places in the group stage. One team could face France and Germany on its way to the final, while another might play Mexico and Greece. With such different opponents, comparing the performance of teams or players would be all but hopeless. And match data from this tournament, no matter how interesting, will almost certainly be skewed by unlikely events -- the kind that can turn a single game but might never repeat themselves in the course of a season. If none of the data is repeatable, it's hard to see how it's descriptive.

So, can analysts at least go back to look at national team performance in qualifying to predict World Cup performance? Not really. For one, national teams play very infrequently, even in a World Cup year. The United States Men's National Team (USMNT), for example, played a total of six matches this year in the lead-up to the tournament -- three fewer than Manchester City played in the month of December.

Moreover, national team rosters change a great deal between international breaks, whether due to injury or experimentation. Eddie Johnson and Landon Donovan were the goalscorers in the USMNT's 2-0 win over Mexico last September that sealed the Americans' World Cup qualification. But neither made the final 23-man roster that went to Brazil. Without the same players on the field, how can you use data from a national team game in November to make claims about how the team will play more than half a year later?

Despite these challenges, most firms have packaged and released World Cup match data, letting fans and journalists take from it whatever they wish. Prozone provides various baubles including individual possession maps; MatchStory offers a team-by-team summary with some basic goal data; Opta gives readers the same data it does for league sides, with some added nuggets here and there for each World Cup team. And Fivethirtyeight has a constantly updated and adjusted World Cup prediction model. Last time around, it managed to explain only 34 percent of the final team rankings -- and that was better than most.

On the one hand, something is better than nothing. A columnist attempting to make a point about team tactics can do no worse than to back it up with some relevant performance data. And several prediction models have beaten the bookies using the same system as chess rankings.

On the other, presenting raw numbers as part of gorgeously packaged interactive infographic, whether tackles or pass completion rates or whatever, gives readers the illusion that the data carry intrinsic meaning.  "Team A tackled more than Team B, therefore Team A is better at defense." "Team X had a lower pass completion rate than Team Y, therefore Team Y is more technically gifted."

Or maybe Team A was defensively lax and therefore forced into more desperate play, and Team X was playing in a more direct but riskier way that entailed fewer successful passes.

This is the kind of equivocation skeptics use to cast doubt on data-driven approaches to sport. It also drives serious sports statisticians crazy. Raw match data is only as useful as what you do with it, whether by running regressions or developing predictive models using a large sample of games. Repeatability and reproducibility is a much harder sell than magical match numbers, but understanding these is what helps transform raw data into knowledge. The World Cup makes that kind of work difficult. Not impossible, just difficult.

That's more a problem for the data analysis firms than it is for fans. Because let's remember what's really important: The very same variation that makes a knockout tournament like the World Cup so difficult to predict is also what makes it so exciting. 

Martin Rose / Getty Images Sport


If Japan Wanted to Build a Nuclear Bomb It'd Be Awesome at It

But let's all take a deep breath: Tokyo may be dumb, but it's not stupid.

So, it seems that Japan lost a little bit of plutonium. Cue the outrage! Well, not lost, exactly. The Japan Atomic Energy Agency (JAEA) submits a voluntary declaration to the International Atomic Energy Agency (IAEA) that documents exactly how much plutonium the nation has stockpiled. For fundamentally clerical reasons, the JAEA accidentally omitted the 640 kg plutonium contained in a load of fuel at a nuclear power plant. The material was never unsafeguarded or misplaced.

Japan is the only non-nuclear weapons state with significant holdings of civil plutonium. Given the historical animosities, Japan's plutonium stockpile is something that understandably makes its neighbors a bit nervous -- maybe even a little crazy. So, for example, when Japanese Prime Minister Shinzo Abe agreed to return some poorly guarded weapons-grade plutonium and highly enriched uranium to the United States that had been used for research, the international reaction was not relief. It was: YOU HAD WHAT?

I am a critic of Japan's policy of separating and reusing the plutonium inevitably created in the country's nuclear power plants. Japan's stockpile of plutonium sets a terrible example for other states like, say, Iran. Still, we should not lose sight of the fact that Japan is not going to build nuclear weapons.

Much of the concern expressed by Japan's neighbors is simply a convenient opportunity to give Prime Minister Abe a kick in the shins. And, frankly, he probably deserves more than a few kicks in areas north of the shins for stunts like visiting the Yasukuni shrine and throwing shade at the women raped by the Imperial Japanese Army.

Yet the notion of Japanese nuclear weapons keeps turning up. The idea has gotten some attention in light of the general combativeness of the most recent International Institute for Strategic Studies' Shangri-La Dialogue (I am trademarking "The Brou-ha-ha in Shang-ri-la"), where Chinese participants acted boorishly, as well as an interesting debate between my friends David Santoro and Elbridge Colby about whether the United States should ditch Asian allies that leave the Nuclear Non-Proliferation Treaty (NPT) in search of the bomb.

These are important discussions, but they give the wrong impression. Focusing on the unlikely possibility of a nuclear-armed Japan distracts from more important policy challenges that threaten the shared interests of the United States and Japan in arms control, disarmament, and nonproliferation.

Don't get me wrong, there will always be a certain constituency within Japan for extremist views. Shintaro Ishihara, the former governor of Tokyo, has made a career out of saying impolitic things, including his infamous book, The Japan That Can Say No -- say "no" to the United States, that is. Ishihara says "yes" to nuclear weapons and a bunch of other terrible ideas, from purchasing the islands at the center of the maritime dispute with China to suggesting that sexual enslavement was "a very good way of making a living" for a young woman in wartime.

There have always been extremists in Japan who aren't one bit sorry about the war. Take Ishihara's buddy, the late Yukio Mishima. Three times nominated for the Nobel Prize in literature, Mishima was also an actor who later in life got into extremist right-wing causes, body-building, and so on. In 1970, he and some of his young acolytes in a student militia called the Tatenokai entered the military base in Ichigaya and exhorted the soldiers to launch a coup to restore the emperor. The soldiers looked on, sort of baffled -- some accounts say they even heckled him -- and then Mishima retired to an office to commit seppuku, or ritual suicide. The plan was that Mishima would stab himself in the stomach and then one of the students, alleged to be his lover, would behead the well-known author. The lop job didn't go exactly as anticipated: The poor fool botched it a couple of times, leaving another student to finish off Misihima. Japanese Prime Minister Eisaku Sato, who knew Mishima socially, said, "I can only think he went out of his mind." So, yes, there are weirdos in Japan. (And elsewhere: Someone pinned Mishima's severed head on Pinterest.)

As an American, I can tell you that it's not fair to judge a country by its nut-jobs. A far larger and more important constituency in Japan are the people who categorize the devastation of World War II as a catastrophe, the post-war reconstruction as a miracle, and the existence of nuclear weapons as abhorrent. This is the Japan of Yellow Magic Orchestra and Hello Kitty. (Oops.)

It's easy to talk about Japan building nuclear weapons, but the real policy debates reflect Japan's nuclear allergy, not enthusiasm. In late 1969, a few months before Mishima killed himself, the United States agreed to return Okinawa to Japanese control. The sticking point between Tokyo and Washington was whether U.S. bases would continue to host American nuclear weapons or not. Ultimately, the United States relented to Japan's demand for an Okinawa without nuclear weapons, although Prime Minister Sato agreed to consult with the United States in the event of a crisis. (Sato is Abe's maternal great-uncle, by the way.) Even that agreement, however, had to be signed in secret. After signing the official memorandum to return Okinawa to Japan, then-National Security Advisor Henry Kissinger and his Japanese counterpart actually contrived for U.S. President Richard Nixon to invite Sato into the president's study to look at some objets d'art so they could sign the secret agreement without anyone present.

This is hardly ancient history. In 2010, when the Democratic Party of Japan (DPJ) finally took sole control of the government for the first time in post-war Japan, it ordered an inquiry into secret agreements like the one Sato signed. (The Japanese copy was found by Sato's son, who would be Abe's first cousin once-removed, if you are keeping score.) The DPJ calculated, correctly, that secret agreements to allow U.S. nuclear weapons to enter Japan would outrage a good portion of the public. The result was an ugly spat between the Obama administration and the DPJ government. The Japanese public, by and large, thinks what happened in Hiroshima and Nagasaki was a terrible thing. I am a member (that's my head!) of the Governor of Hiroshima's roundtable on disarmament. Let me tell you, nuclear weapons are not a vote winner in Japan.

Nor, I hasten to add, is Japan "six months away" from a bomb, even if you hear that all the time. Recently some senior U.S. officials repeated the "six month" claim to NBC reporter Robert Windrem. There is no technical basis for such a statement. I once actually tried to trace the heritage of that irksome claim. As far as I could tell, it dates to a conversation with a "Japanese strategic thinker" in 1976 that is cited in Richard Halloran's 1991 book, Chrysanthemum and Sword Revisited: Is Japanese Militarism Resurgent? The claim is made in passing, not as a formal assessment of Japan's technological capability or plans. "Six months" in context is like the biblical "40" -- that is to say, it means "fairly soon" just like "40 days and 40 nights" means it rained a long, long time.

Other than one bit of yellow journalism in the Sunday Times, no one has attempted to document a technical basis for the "six-month" claim. There have been several Japanese and American assessments, from academic studies to declassified intelligence reports, on the possibility that Japan might build nuclear weapons. All of them conclude that a nuclear deterrent would cost Japan a few billion dollars and would take several years to build.

That's because the Japanese would not jury-rig a tiny arsenal out of civil plutonium. They could do it, sure, but why? Why completely alter the structure of Japanese security policy for a handful of makeshift bombs that might not work? If Japan goes nuclear, it will do so only as part of a fundamental change in how the Japanese look at their security environment. In that case, Japan would build nuclear weapons like they do everything else, down to the beer machine at Narita -- with meticulous care. Japan would construct dedicated plutonium production reactors and facilities to separate weapons-grade plutonium, probably conduct nuclear tests, and deploy modern delivery systems, such as missiles.

This is, I would argue, the most important point to understanding U.S.-Japan relations, and extended deterrence. We often talk about nuclear weapons in Japan like a thermostat -- if U.S. credibility declines in Tokyo, Japan will build a nuclear arsenal to compensate. It's almost as if we cut 10 bombs, the Japanese will want 10 of their own to make up the difference. That's not right at all. For Japan, becoming a nuclear weapons power would require a dramatic break in a foreign and security policy that has historically centered on the U.S. alliance. So would unarmed neutrality. It is Japan's lack of such strategic options that account for the most interesting Japanese behaviors in foreign and security policy.

As one Japanese observer pointed out to me, neither alternative -- nuclear-armed independence nor unarmed neutrality -- has a mainstream constituency in Japan. That means the only practical approach for Japanese policymakers is an alliance with the United States. Tokyo has little choice but to accept whatever level of security Washington can provide at the moment. Another colleague compared it to riding on the back of a motorcycle -- you can see the bumps and twists in the road, but you can't do anything about it. That's scary. The result, of course, is a lot of whining from Japan about the credibility of the U.S. guarantee. What else can they do? And it accounts for the tendency of the country's politicos to fixate on symbols of Washington's commitment, just as Max Weber observed that Protestants tended to obsess about material success as a sign of predestination.

This is the downside of kicking Japan over its stockpile of plutonium. Abe's historical revisionism is about making Japan a victim, rather than a contrite member of the international community. Yes, Japan's plutonium policies are an unwelcome precedent for countries that don't have its same allergy to nuclear weapons. But accusing the Japanese of harboring a secret ambition for nuclear-armed militarism only reinforces Abe's nonsense about foreign criticism of Japan and makes it harder for sensible Japanese voices to push back. It exacerbates the underlying anxiety that the United States will abandon Japan -- only making it harder for the two countries to work together to beat back the bomb.