There are two ways to get those answers. Most often we guess. But those guesses are based on data that can mislead us utterly. People still take the answer seriously, says Ball, "even though the science underneath it may not be any better than in the '60s and '70s, when we just made the stuff up. It's not only easy to misinterpret these numbers; it's substantially easier to misinterpret them than to interpret them correctly. You look at the graph and say, 'Now I know what happened in Liberia.' No, you know what happened to people who talked to the truth commission in Liberia. You can say there were at least 100 people killed. But you can't say we had five in the north and five in the south and start drawing patterns."
Ball's accomplishment has been to provide an alternative to guessing: With statistical methods and the right kind of data, he can make what we know tell us what we don't know. He has shown human rights groups, truth commissions, and international courts how to take a collection of thousands of testimonies and extract from them the magnitude and pattern of violence -- to lift the fog of war.
IN THE LATE 1990s, Ball was commuting between Cape Town and Guatemala City, working for both the South African and Guatemalan truth commissions, immersed in the varying atrocities of apartheid and genocide committed by two very different regimes. Then in June 1998, when the staff of the South African Truth and Reconciliation Commission was writing its report, commissioner Mary Burton asked him what should have been a simple question: How many killings had there been in apartheid South Africa?
"According to the testimonies, 20,000," said Ball.
"No," said Burton. "How many were there in all?"
Uh-oh, thought Ball. "I don't have an answer," he told Burton.
The commission had been hobbled by a boycott from the Inkatha Freedom Party, the second-largest black political group, which had blocked its members from testifying because its leaders thought the commission was biased toward the African National Congress, with which it was involved in a long, simmering war. But just before the deadline for testifying expired, Inkatha lifted the ban and its members came forward. Those testimonies completely rewrote the commission's conclusions, as some 8,000 Inkatha members gave evidence -- about one-third of the commission's final total. What, Ball wondered, would have happened if Inkatha hadn't decided to participate? "Are people really coming forward in the same proportion as they suffered?" he asked himself.
As he was talking to Burton, it hit him: We don't know what we don't know. "It wasn't enough to take good care of the evidence we have," he says. "We have to transcend what people are willing to tell us. You're thinking about: What can I fix about the data coming in? But the big problem is: What's true in the world?"
At the University of Michigan, where Ball studied sociology in a program heavy on statistics, the solution to such problems was clear: Go out and take a random sample. You choose households at random and survey them about what happened. Since your sample is representative of the whole, you can easily extrapolate the results to the larger universe. But this was not something human rights groups knew how to do, and it would have been prohibitively expensive. It was not the answer.
Working on the Guatemalan data, Ball found the answer. He called Fritz Scheuren, a statistician with a long history of involvement in human rights projects. Scheuren reminded Ball that a solution to exactly this problem had been invented in the 19th century to count wildlife. "If you want to find out how many fish are in the pond, you can drain the pond and count them," Scheuren explained, "but they'll all be dead. Or you can fish, tag the fish you catch, and throw them back. Then you go another day and fish again. You count how many fish you caught the first day, and the second day, and the number of overlaps."
The number of overlaps is key. It tells you how representative a sample is. From the overlap, you can calculate how many fish are in the whole pond. (The actual formula is this: Multiply the number of fish caught the first day by the number caught the second day. Divide the total by the overlap. That's roughly how many fish are really in the pond.) It gets more accurate if you can fish not just twice, but many more times -- then you can measure the overlap between every pair of days.
Guatemala had three different collections of human rights testimonies about what had happened during the country's long, bloody civil war: from the U.N. truth commission, the Catholic Church's truth commission, and the International Center for Research on Human Rights, an organization that worked with Guatemala's human rights groups. Working for the official truth commission, Ball used the count-the-fish method, called multiple systems estimation (MSE), to compare all three databases. He found that over the time covered by the commission's mandate, from 1978 to 1996, 132,000 people were killed (not counting those disappeared), and that government forces committed 95.4 percent of the killings. He was also able to calculate killings by the ethnicity of the victim. Between 1981 and 1983, 8 percent of the nonindigenous population of the Ixil region was assassinated; in the Rabinal region, the figure was around 2 percent. In both those regions, though, more than 40 percent of the Mayan population was assassinated.
It was the first time anyone had applied MSE to human rights work. "He produced numbers that provided a strong, crisp basis for drawing the conclusions the commission did about violence, in a way you can't get from testimony," says Kate Doyle, a senior analyst with the National Security Archive at George Washington University, who has worked extensively with archives in Guatemala.
SUBJECTS:
(42)
HIDE COMMENTS LOGIN OR REGISTER REPORT ABUSE