I agree with 2GEORGE that death-tolls somehow miss the moral point. Both suffering and morality are personal, and nothing is more impersonal than numbers. Yet Ball's use of numbers plus analysis occasionally is able to reveal patterns that bear on intent, as apparently happened in the Yugoslav case, and then you are approaching the universe of moral discourse. So I would not dismiss his efforts altogether. Having read only the cited paper's abstract, I look forward to reading the rest.

Ball's report was rejected by the Tribunal. Dr. Eric Fruits totally discredited Ball's methodology and his conclusions regarding Kosovo.

very good web: === http://www.plzzshop.com

The website wholesale for many kinds of fashion shoes, like the nike, jordan, prada, also including the jeans, shirts, bags, hat and the decorations.

All the products are free shipping, and the the price is competitive, and also can accept the paypal payment., After the payment, can ship within short time.

We will give you a discount

WE ACCEPT PYAPAL PAYMENT

YOU MUST NOT MISS IT!!!

=== http://www.plzzshop.com

thank you!!!

Believe you will love it.

We have good reputation, fashion products,

come here quickly== http://www.plzzshop.com

Opportunity knocks but once

I did comment on the Bosnia mortality statistics, and my comments were specifically in favor of the enumeration by the Sarajevo-based Research and Documentation Centre that found approximately 97000 deaths (see http://www.hicn.org/research_design/rdn5.pdf). The RDC's enumeration was subsequently validated as nearly complete by an MSE estimate done by ICTY demographers Jan Zwierzchowski and Ewa Tabeau (I think their excellent study was published only at the Feb 2010 Households in Conflict Network conference in Berlin, but it might have come out elsewhere). If I understand your comment correctly, it seems that we largely agree on the total mortality for the Bosnian conflict.

The errors can go both ways, of course. The problem is that the matching may either undercount or overcount the number of matches because of errors of memory. Also, the circumstances are precisely those that produce false impressions. When bombs fly & people are being killed, your activities as a witness are primarily to do what is needed to avoid becoming a corpse. Your attention to everything else is as small as it ever will be. Whereas normally, you might well be expected to remember the names of those who perished from natural causes with accuracy (that's questionable in itself), in a war time situation, any assumption of that sort cannot hold eo ipso.

No matter how elegant the mathematics of such computations are, they cannot correct for inaccuracies of persons reporting the deaths. Like it or not, only exhumations can produce accurate body counts.

The real key, before accepting Prof. Ball's method for future events, is to see how well it works against a gold standard. When one knows in actuality the number of deaths in a calamity, war or famine being two examples, one could then apply the method & see if it holds up. Otherwise, there is no reason to trust the numbers of deaths. There is, by the way, no reason to accept this analysis simply because it had been done before, perhaps with great inaccuracy, in Nicaragua.

Robert Conquest used aberrant mathematics to estimate Stalinist deaths. The use of his method would show the US during the depression killed even more.

If his methodology seems odd it's because it's bogus.

Go to icr.icty.org and download Dr. Fruits' refutation of Ball's findings. Dr. Fruits' expert report is exhibit 3D00893 in the Milutinovic et al trial. He lays it all out and exposes Ball for the fraud that he is.

The Tribunal rejected Ball's findings, his work basically amounts to garbage in = garbage out. His data was flawed, his methodology was flawed, and his conclusions were unsound -- and the trial judgment says the judges didn't even trust his objectivity as an expert.

It's tough to have a gold standard, but there are two ways to come at it: first, through simulations. We can simulate in a computer random populations, then draw samples (even very biased samples) from the simulation, then make estimates using MSE from the samples. One way to address your concern about the difficulty of uniquely identifying people is to add error to the simulated linkages. In this way, we can assess the impact of identification error on the estimates. I've done this for a couple of projects, and the changes in the estimates tend not to be sufficient to affect interpretation. That is, in most real-world situations, overlap measurement error does not seem to contribute substantially to estimate bias.

Another question you raise is the independence of samples. This is a key point, and in the MSE literature it is addressed as "list dependence." The solution is to use more than two lists, and then estimate the correlation between subgroups of lists, thus correcting for list dependence. The classic source on this method is Bishop, Fienberg, and Holland, _Discrete Multivariate Analysis_, ch6. In Kosovo, we used 4 lists.

Another approach is to look at a real-world situation with multiple methods. Kosovo March-June 1999 offers such a case. There were two probability-sample surveys (one done by Physicians for Human Rights, the other by a group from the CDC led by Paul Spiegel and Paul Salama); the MSE that my colleagues and I did; and recently, a near-census of all killings done by the Belgrade-based Humanitarian Law Centre. The MSE and the surveys agree closely with an overall estimate of total killings March-June 1999 of approximately 10,000. The surveys do not offer enough detail to test the time and space patterns. The HLC near-census matches the time and space patterns found in the MSE remarkably closely. The data for the HLC and the MSE estimates I did is all online, it's easy to check out. A similar validation has been done with a near-census and MSE in Bosnia.

Whether this example validates MSE in general for conflict-related mortality is an ongoing debate, but I think evidence from multi-method approaches -- in Kosovo, in Bosnia, and in Timor-Leste -- is accumulating in favor.

"I think evidence from multi-method approaches -- in Kosovo, in Bosnia, and in Timor-Leste -- is accumulating in favor." -- Therefore it is?

so the method, which is very odd to me, lacks any real proof of validity. Robert Conquest estimated excess deaths under Stalin by the use of two census figures; complete garbage. Let's go through your methods, if we might.

In the statistical measurement of social phenomena, there are three steps in validation. First, does the math work in a formal sense, given relevant assumptions? Second, does the estimation agree with known-good ("ground truth") simulations, and with how much sensitivity to perturbations of the assumptions? Third, when there are multiple measurements of a real-world case of the phenomenon using different methods, do the methods agree?

MSE has passed the first test; there is a wide and growing literature in demography, ecology, and biological and mathematical statistics. Evidence from the second test is emerging: there's a lot done in the academic literature, and there are several studies that I know of (some by me and my colleagues) looking specifically at the conditions we expect in the measurement of conflict-related mortality. The third test (multiple measures) is expensive and consequently rare. In the three cases we have, MSE has been found consistent with the other measures.

Certainly much more can and should be done. A number of scholars have posed hypothetical situations in which they believe MSE would fail, and I am actively engaged in exploring the implications of those concerns. So far, however, all the empirical results are in favor of MSE's validity.

and not on "one" side, if you know what I mean. There was this pair that claimed they could prove by virtue of statistics that the Iranian election was fraudulent. If you knew about number crunching it became apparent fairly quickly that they were taking the results, hunting for a weird pattern (they always exist) & then doing a statistical analysis on it.

In this case, the abuse of statistics is worse because it ignores vital differences between fish who are labelled by humans & then counted & reporting from distressed humans who almost by definition will provide inaccurate reports. Virtually every one reading this can recall a minor emergency they experienced when they were certain something happened which did not actually occur or did not see something obvious going on that everyone else noticed. Now imagine how accurate you would be in a war zone, when your life & the lives of your family & friends are at risk. Don't you think you just might get details wrong, like who had actually died? All you care about is saving the living.

The ICTY findings explicitly found that I was *not* biased: from the section that you cite above, para 24, "His evidence will therefore be examined below on a substantive basis."

You're right that the trial chamber did not accept my evidence, but I think they erred in doing so. Dr Fruits' primary objection was that we did not include Yugoslav force activity in our analysis. True, but we did not make conclusions about Yugoslav force culpability -- we were doing something different, which was rejecting the possibility that NATO or KLA were plausible causes of killing and migration. Dr Fruits managed to confuse the judges on this point, but of course that was his job as a defense expert, after all.

Dr Fruits further criticized our findings by raising a number of tests that he thinks we should have done; he did not conduct those tests himself. After reading his report, we did some of the tests he suggested, and we found that the tests confirmed our original findings. Unfortunately for the debate, the prosecutor chose not to present this in cross-examination. The problem is that there are so many possible tests, not all can be done in any study. We did the ones that we felt were most relevant. Of course, as Dr Fruits noted, there are more tests that could be done, but the judges did not seem to understand that Dr Fruits was raising essentially redundant and irrelevant concerns. Alas, but this is the risk of legal vs scientific debates.

As noted in the article above, the Yugoslav authorities announced a unilateral cease fire on Tuesday 6 April 1999, and they maintained their cease fire until Saturday 10 April (the cease fire stopped the day before Orthodox Easter). The Yugoslav authorities did close the border, as you note. During this period, KLA and NATO activity increased, doubling and tripling relative to the previous week. However, and this is the key point, *killings* declined from their second-highest point in the conflict 1-6 April to nearly zero during the 7-10 April cease fire. When the Yugoslav forces announced that they were resuming operations, killings increased rapidly to the third-highest peak on 14-15 April.

The argument is that if the KLA or NATO were responsible for killings, we would expect that their increased activity during the 6-10 April period would be associated with an increase in killings; in fact, the opposite occurred. When the Yugoslav forces publicly announced that they were suspending operations, killings declined to near zero, and when the Yugoslav forces announced that they were resuming operations, killings increased. This isn't proof that the Yugoslav forces committed the killings, but, as I said in both Milosevic and Milutinovic, we observe only that this correlation is consistent with the claim that the Yugoslav forces were responsible for the killings. The report is here: http://shr.aaas.org/kosovo/icty_report.pdf

Finally, Tina and I discussed the Milutinovic case at length, but she chose not to include it in her article. Not my call. Also the photo was taken by FP's photographer, he conceptualized it and set it up, and FP editors chose this photo in preference to more traditional images. Again, not my call.

How can you say otherwise? What proof have you that this technique actually works from a test against a gold standard in human reports from a calamity?

Mr. Ball,

I can forgive you for the picture. That could have happened to anybody.

But para 24 of the judgment says, "The Chamber considers that the evasive nature of the witness’s responses casts doubt upon his objectivity as an expert witness. However, Ball’s expert reports do not, on their face, display any signs of bias in respect of their preparation and contents. Moreover, Ball displayed no bias during his oral testimony before the Chamber. His evidence will therefore be examined below on a substantive basis."

If they explicitly found that you were not biased, then what did they mean when they said: "The Chamber considers that the evasive nature of the witness’s responses casts doubt upon his objectivity as an expert witness." ?

And when you say, "Tina and I discussed the Milutinovic case at length, but she chose not to include it in her article." Does that mean that you told her that your evidence had been rejected by the Tribunal? In your lengthy discussions, did you ever tell her that?

Lying by omission is still lying. If you told her that your evidence was rejected, and she chose to withhold that information from her readers, then she's dishonest. If you never told her, then you're dishonest. Which is it, did you tell her or not?

The substantive parts of para 24 say that my expert reports and oral testimony do not display bias; this seems to me the conclusion. The part the court didn't like was a line of questioning in which, frankly, they confused me and, in the moment, I couldn't figure out what they were getting at. I disagree that I was evasive.

When I discussed the case with Tina during our interview, I told her that the court rejected the evidence, and I explained why I think they erred in their judgment. It would be interesting to revisit the technical debate in a scientific forum moderated by statisticians. Perhaps AAAS or Am Stat Ass'n could organize a panel.

I owe you an apology if you really did tell Ms. Rosenburg that your evidence was rejected. I'm going to e-mail her and see what she has to say about that, because if you were honest and you told her, and she concealed that information from her readers, then she's not a very credible journalist is she?

I also note that para 24 doesn't exactly give your reports the good housekeeping seal of approval. The judgment isn't as definite as saying that your reports are not biased. It says, "Ball’s expert reports do not, on their face, display any signs of bias in respect of their preparation and contents."

By using the qualifier "on their face" they left open the possibility that your reports are biased. What they're doing in para 24 is they're giving you the benefit of the doubt. Eventhough they felt that you had testified in an evasive manner and they had doubts about your objectivity as an expert, they nonetheless evaluated your evidence on a substantive basis, rather than dismissing it outright, because they did not detect signs of bias in your work itself -- at least not on its face.

The reason why they rejected your evidence was because they felt that Dr. Fruits had successfully undermined it. His report is exhibit 3D00893 in the Milutinovic trial, I encourage anyone reading this to go to icr.icty.org and download a copy of it. I'm not a scientist, but it appears to me that his critique of your work goes much deeper than you've indicated here.

Your remark that "Dr Fruits managed to confuse the judges on this point, but of course that was his job as a defense expert, after all." is telling. How did you view your job as a prosecution expert? Was it your job to confuse the judges in order to secure a conviction for the prosecutor?

Journalists do not appear in courtrooms. All they do is report information to the public that may or may not be accurate. Sometimes, as in the present case, they do not completely research the subject. These are common flaws. Incomplete research is characteristic of all of us, journalists or otherwise, because there's always stuff out there we did not read.

In the case of statisticians who blow it, often they believe in what they are doing. They are simply wrong. Again, this is characteristic. Which of us can say we have never been wrong? Sometimes, of course, as with the Iranian election pair whose work appeared in the Washington Post, that is impossible to believe. The reason is that they used different means of assessing the election results in Iran than they did in their prior study, rendering reasonable the inference of post-diction (e.g., looking for the result in the data that would yield P < 0.05).

Failure in a courtroom is far worse than failure in a journal, magazine, or newspaper. Prof. Ball's error was infinitely more serious than Ms. Rosenberg's mistake. The reason to think his failings were not simply the being wrong that afflicts everyone is that he appears to have used the same flawed method in subsequent cases. Unless he admits he was wrong & changes his methods of analysis, he should never again be listened to by anyone.

Her "mistake", like Dobbs's "mistake" was to recite fraudulent claims and stamp them as true.

@ Martial - I do attribute Ms. Rosenburg's error to incomplete research. It certainly appears as though Patrick Ball misled her when she interviewed him. It appears that he held up his work in Kosovo as a shining example of his expertise and, in the hopes that she would write a favorable article about him, neglected to tell her that his evidence was rejected by the Tribunal because his data and methodology were found to be unsound and the judges doubted his objectivity.

He told her point blank, "People who want to dismiss us say we're just making s**t up. If they're ever right when they say that, we're in big trouble." Clearly, he led her to believe that his work was sound and that it had withstood scrutiny when, as we can see from the judgment and the evidence in the Kosovo 6 trial, it hasn't.

The fact that he would mislead her, and do it in such a brazen way, says something disturbing about his mental state. The fact that he could look her in the eye and make a cocky remark like that when he knew that his work on Kosovo had been discredited and rejected by the court -- it says a lot about the kind of person he is.

I think he fooled her and took advantage of her in a cynical effort to enhance his own reputation. Ms. Rosenburg's error can be forgiven if she goes on record and sets the record straight. If she publishes a retraction saying that he that he misled her, then she can be forgiven. But if she doesn't do that, if she chooses to let this deceptive article stand, then she's as guilty as he is. A person like Patrick Ball can't be trusted to work in the field that he's in. He needs to be exposed.

my realm of expertise. You may be right; on the other hand, Foreign Policy & journalists in general may consider the printing of uncontested contradictory argument sufficient. Comments sections are vital, nitpicking, petulance, & dishonesty of commentators notwithstanding; they help ensure journalistic integrity.

Online publications that most bruit "Zionist control" oft least permit comments | most police them. Consider Counterpunch. http://www.counterpunch.org/2012/02/22/what-really-happened-in-the-yom-kippur-war/ , which regularly posts Mr. Israel Shamir, in reality Norwegian neo-Nazi Mr. Joren Jermas http://my.telegraph.co.uk/updates/tag/joran-jermas/ Neither Counterpunch nor Mr. Jermas' website http://www.israelshamir.net/English/Putin.htm permit comment.

is not a Nazi concept as most would conceive of it, rather it is a tactic the Nazi's alleged that Jews used to control the planet. Prof. Bytwerk, as usual, has words concerning this that cannot be surpassed:

Hitler and the “Big Lie”

The false Goebbels quotation above is actually a take-off on Hitler's familiar statement in Mein Kampf, which is often misunderstood. Hitler stated:

“In this they [the Jews] proceeded on the sound principle that the magnitude of a lie always contains a certain factor of credibility, since the great masses of the people in the very bottom of their hearts tend to be corrupted rather than consciously and purposely evil, and that, therefore, in view of the primitive simplicity of their minds, they more easily fall victim to a big lie than to a little one, since they themselves lie in little things, but would be ashamed of lies that were too big. Such a falsehood will never enter their heads, and they will not be able to believe in the possibility of such monstrous effrontery and infamous misrepresentation in others.…” (p. 231 of the Manheim translation)

Hitler is accusing the Jews the Vienna press of this strategy. It is often taken as evidence that Hitler advocated the “Big Lie.” He is, in fact, accusing his enemies of lying.

Now, Hitler was entirely willing to lie — but in public he insisted that he and his propaganda were truthful.

http://bytwerk.com/gpa/falsenaziquotations.htm

He is a wonderful man, Prof. Bytwerk. A point that I always make because of people like him is that no German or Japanese born after say, 1935, is responsible in the slightest for fascist atrocity. I once begged him to translate all the Nazi propaganda films, but he said that had already been done. Wish they were online!

but it's definitely there, fully documented. You can find it if you look. Appearances can be quite deceiving; some women reported feelings of great comfort & safety when being escorted to cars by Mr. Ted Bundy. My slightly more generous approach to Prof. Ball than a colleague here may stem from abandoning Wilcoxon's T tests for the bootstrap, yielding greater sensitivity & fewer assumptions.

I am not the author of the quote you attribute it to me. You are confusing me with Tina Rosenberg.

One may use proofs, as with the Nazi numbers provided by the Nazis, to determine numbers of deaths. What one should avoid is uncorroborated accounts of refugees, who, being in grave distress, are very likely to remember with certainty that which did not occur and also to fail to remember that which was rather obvious. Again, consider your own life, with minor emergencies, not a battle field, & see if you do not find the above to be the case.

Also, corpses exhumed are different objects than witnessed deaths. Essential would be checking to see if a different modeling procedure was appropriate for the analysis of corpse counts than of witnessed deaths.

The point of the paragraph is to say that although only 4,400 deaths were documented by name, we believe that there were more deaths that had not (at that point) been documented. This is one of the various estimates we made of the total deaths. (we made a series of estimates using slightly different methods, and by stratifying time and space in different ways)

I entirely agree with you that exhumations are an important way to document war deaths, and exhumations were one of the four data sources we used in the estimates. There were about 4000 exhumed cadavers when my colleagues and I were doing this research in 2000-2001. Of those, approximately 2,000 had been identified, if I recall correctly. The exhumations proved that there were at minimum 4,000 violent deaths (there have been more exhumations in Kosovo and in Serbia since I got the data in 2001, but I don't have the newer data).

The statistical work showed the larger pattern, including deaths not documented, the pattern of those deaths over time and space, and the relationship of the time-space patterns with time-space patterns of other phenomena (migration, NATO airstrikes, KLA clashes with Yugoslav forces).

Thus I think the forensic and statistical work are complementary, addressing different questions.

fish labeled by human scientists are not corpses in the ground, let alone deaths reported by refugees. No mathematical manipulation of unreliable observations achieves reliable conclusions. You render equivalent exhumed body counts & reported deaths; no statistical analysis properly assesses a chimera. All material submitted for use by a criminal trial, which yours was they say, is forensic by definition.
Another point.
ABA “All interviews were conducted using a standardized questionnaire that allowed for a narrative description of events. . . .For the statistical purposes of the present study, all of the data were recategorized from the original database into new data structures. All data were recoded from their original formats into standard geographic classifcations and date precision codes.”

HRW “All interviews were conducted to elicit open narratives of what the interviewee had seen. Standardized questionnaires were not used (HRW 2001). Despite not having used a standardized questionnaire, the interviews were rich sources of information about killings. They were coded and entered into a database. Coding for the present study was independent of the original HRW database and the statistical work presented earlier in HRW.”

OCSE “OCSE-KVM used standardized interview forms similar to those used by ABA/CEELI. The information was then entered into a database, also similar to that used by the ABA. For our study, the data were independently recoded, as we did for HRW. The semi-structured OCSE interviews were reformatted to be compatible with the format we developed for use with the ABA/CEELI data.”

Differences in data acquisition & handling render likely a person’s being declared unidentified by one system & named by another; inflating the number of unique names overestimates total deaths. This is not a statistical practice Fisher would have applauded.

I share your concern with unidentified deaths failing to match identified deaths. Therefore we excluded all unidentified deaths from the lists used for estimation; there's a section toward the end of Appendix 2 about what the patterns of unidentified deaths look like relative to the identified ones.

However, only records that had at least first and last name were included in the matching and the estimation. There were of course many slight variations in place and date of death, as well as name spellings. We assumed whenever the records might plausibly match, that they did in fact match. This assumption probably biased downward the total count of observed deaths (because it maybe have counted only once two unique deaths), and biased downward the estimate of total deaths (because it increased the overlap between systems).

still be a problem. Were dead persons A,B,C,D are named in list 1 and not in in list 2, 3, or 4, even though they were seen, then there exist four falsely unique names even when unidentified persons are excluded.

Good Sir, I do not wish to cause distress, but must tell you your approach has problems. Wartime civilian corpse counts, having limited accuracy & precision, are perhaps better viewed as perfume to be sniffed than Thunderbird to be guzzled (always get bleary eyed over the stuff). At the very least, it can be said you made a diligent effort at a noble task, one you believed strongly in.

@MARTIAN,

I think your objections are overblown. You raise some valid points but you're confusing some things. First, what makes "garbage" a dataset is not how it was created, but attributing it properties that it doesn't have. The assumptions of MSE in this sense are surprisingly mild, allowing you to use data that would otherwise be considered "garbage".

It should be clear by now that MSE methods do not need unbiased or representative samples from the target population to work. What they need are just enough samples so that the joint inclusion mechanism, which includes biases and induced correlations, can be reasonably approximated and estimated. Of course, what "reasonably approximated" means is rightly debatable. However "debatable" does not mean "impossible".

There's a conceptual confusion in your discussion of unidentified deaths. MSE methods do not work by modeling deaths but reports of deaths. The detail is how you define a "report". In the Kosovo work, to call something a "report" they required name and surname. This means that if in one of the projects there was a mention of an unidentified death, that was considered a no-report. So, if "dead persons A,B,C,D are named in list 1 and not in list 2, 3, or 4, even though they were seen", there *do not* exist "four falsely unique names", as you imagine. What you have are 4 individual records with a pattern of being listed with name and surname (or "reported") in list 1, but not in the other lists. This is data you can work with. Under that framework, the question that MSE answers *is not* "how many casualties weren't mentioned in the lists?", but "how many casualties weren't listed by name and surname in any of the lists?". Subtle but important difference.