14: Effectively Seeking Funds for a Defense
Statistical Expert: Fact-finding in the Face of Uncertainty
It has been seen the in courtrooms across the nation: The DNA expert tells jurors that the chance that the blood found at the crime scene belongs to someone other than the defendant is less than the chance for winning the lottery. Faced with such damning evidence, what is a defense attorney to do? Even an attorney with some background in science is not qualified to do the complicated statistical analysis involved. Consequently, that attorney's cross-examination of the expert leaves much to be desired. Too often, attorneys finds themselves grappling with the myriad of numbers used by the expert, searching for some way to communicate the limits of the findings, while the jurors become increasingly supportive of the expert's testimony.
Another situation too often seen in American courtrooms: the defendant, a black male, has been indicted by a predominantly white grand jury taken from a majority black community. Defense counsel knows that she can have the indictment dismissed if she can prove the dis-crimination. However, discrimination is an ostensibly random process and is almost impossible for the attorney to prove on her own. Even attorneys with some background in statistics are simply unable to do the complicated analysis needed to yield accurate - and believable - results.
In these situations, defense attorneys must seek the assistance of an expert in statistics. When the client is indigent, securing an expert means convincing a court to provide funds. This article to provides attorneys a framework to convince a judge that statistics can assist the reliability of the fact-finding process, and that in the interests of ensuring that reliability, two statisticians are almost always better than one. Convincing the trial court of those basic facts will put an attorney well on her way to receiving the expert funding for a defense statistician her client needs to effectively present the case.
What is the field of statistics?
In the beginning, science was simple. You observed an event, and tried to explain it. End of story. Thus, Aristotle could tell the world with absolute conviction that there were only a finite number of elements, because those were the ones he could see. In other words, the scientific process consisted of two steps: empirical observation and hypothesis. See Bernard Dixon, The Science of Science: Changing the Way We Think 8-9 (1989); Isaac Asimov, New Guide to Science 6-10 (1984).
This method proved unsatisfactory. Our observations would differ from our hypotheses when the circumstances changed. So, a new element of the scientific process evolved: experimentation. Experimentation works on a basic principle of logic: in order to prove that a particular event or condition was caused by one factor, one has to exclude all the other possible factors. Thus, when conducting an experiment a scientist manipulates her environment to account for all possible factors. Only then will she be able to exclude those factors from con-sideration. See Dixon, supra, 12-15; Asimov, supra, 12-14.
When experiments were simple, this was easy to do. Galileo proved that gravity affected all things equally by dropping rocks from a tower. However, as experiments became more complicated, it became harder for scientists to manipulate their environment to the point that they could draw reliable conclusions from their data. Thus, the need for statistics.
Statistics tries to solve a number of problems. First, statisticians try to account for and cor-rect error in the measurement of data. Measurement errors often arise in the context of surveys. For example, scientists studying the relationship between dietary cholesterol and cholesterol levels in the blood neglected to account for smoking when taking their data. As a result, it was unclear whether the results of that data were attributable to dietary cholesterol, or to the fact that many people with bad diets also smoke. Statisticians were able to partially offset that error. Consequently, medical scientists were able to draw meaningful conclusions from what would have otherwise been hopelessly unreliable data.
Second, statistics tries to compensate for extrapolation error. Particularly when dealing with social science experiments, or other experiments on human conditions, it is impossible to gather a complete data set. Without statistics, it would be impossible to determine the extent to which the results of that data set could be used to predict results from subjects outside the data set. If you took a survey of 400 people, and all you knew was that, if the election were held today, 40% would vote for A, and 60% would vote for B, you could not use that to predict who the voters in that jurisdiction would actually vote for. It could be that all 400 of the participants were residents of a small republican enclave in rural Virginia. Or, they could be Hutu tribesmen in Rwanda, who are neither eligible to vote, nor familiar with the candidates. However, if you applied statistical standards, such as the likelihood that the participants would vote, or the geographic and demographic makeup of the sample, you can use that small sample to predict the result of an election in a nation of 250,000,000.
Statistics accomplishes this useful function because statisticians are willing to admit that error is part of the process. The reason this is useful is fairly evident. The standards for absolute scientific proof are almost impossible to meet. For something to be a scientific fact, it must always be true. However, statistics utilizes probability to allow scientists to draw meaningful, useful conclusions in circumstances where absolute truth is impossible. It redefines "proof" so scientists can utilize information that would previously be considered useless. Thus, scientists can treat certain conclusions as facts, without having to succumb to the standards of scientific proof which are impossible to meet in the real world. In short, statistics is a scientific process which permits scientists to make meaningful guesses and use them.
However, that statement does not cover one of the fundamental problems with statistics and the law. The fact that statisticians apply the scientific principles of probability to permit meaningful conjecture does nothing to explain the process of statistics. It merely explains why statistics is useful. The process, or art, of statistics uses scientific and logical principles to decide the relationship between relevant factors. Logically, it is easy to see why these relationships cannot be proven, since it is the uncertainty in these relationships which creates the need for statistics in the first place. Thus, the use of assumptions is implicit to statistics. These assumptions are "gap fillers" which fill in the missing facts which would, in the absence of statistics, prevent scientific conclusions. Statistics is therefore the rules for assumptions, i.e., the use of scientific, mathe-matical and logical rules governing when, where, and how assumptions can be used.
Statistics is both the science of conjecture and the art of assumption.
Statistics is beneficial because it allows us to draw conclusions in situations where the standard scientific process is unavailing.
Statistics is imperfect because it requires statisticians to make difficult and often counter-intuitive choices, without a completely solid factual framework upon which those choices can be based.
Statistics is useful because it enables us to find answers where there are no hard and fast answers. At the same time, statistics is dangerous precisely because of the nature the problem it was developed to overcome, i.e., the lack of hard answers - creates uncertainty and permits abuse.
The Science: How statistics can be used in the courtroom
In United States v. Shonubi, 895 F.Supp 460 (E.D.N.Y. 1995), the court, in a lengthy opinion, discussed the value of statistical evidence in fact-finding. Statistical evidence presented to the sentencing judge in a heroin smuggling case as to the probability that the defendant carried between 1,000 and 3,000 grams of heroin was held to be admissible. The court strongly expressed the value of statistical evidence stating that:
Case law teaches us these important lessons: 1) courts and jurors regularly use statistics to solve legal problems; 2) courts and jurors regularly listen to statisticians from both the prosecution and the defense to make reliable decisions; and 3) to obtain funds to employ a defense statistician, a substantial threshold showing is necessary.
Funds Granted for Defense Staticians
Funds have been granted in Kentucky cases at the trial and district court level for the employment of defense experts to do statistical analysis in a number of cases:
Commonwealth v. William Stark, Jr., Indictment No. 90-CR-11, Shelby Circuit Court, for change of venue survey for Shelby and Franklin Counties in the amount of $3,400. The expert was Walter Abbott, Ph.D.
Commonwealth v. Mark Dunn, Indict-ment No. 95-CR-36, Garrard Circuit Court, for a change of venue survey for Franklin, Garrard and Jessamine coun-ties in the amount of $2,500. The expert was Bruce J. Rose, Ph.D., Kentucky State University.
Kordenbrock v. Scroggy, Civil No. 86-186, federal district court, Covington, $1,463 for the statistical analysis of the underrepresentation of young persons in the grand and petit jury pools. John B. McConahay, Ph.D., Duke University was the expert.
1) The Study Design. Entails identifying the issue to be studied, assessing the nature of the information required, determining how the data will be collected, creating instruments for collecting the data, and defining a sample. Focus on whether the study documents what needs to be proven.
2) The Sample. Is the sample size adequate?
3) Underlying Assumptions. Are there any flaws in the design assumptions?
4) Data Collection. The expert must properly supervise and control the persons involved in the gathering of the data. Survey data must be collected in a non-biased fashion.
5) Analysis. The selection of a statistical method for the purpose of analyzing the data.
6) Computerized Analysis. Determine whether: The data were properly recorded on the forms used. The information was entered into the computer accurately. The computer program designed to manipulate the data was correctly written and implemented. The computer processed the information properly.
7) The Results. Summary of the conclusions.
8) The Limits of Statistics. While the results will reveal whether there is a relationship between the data, they will not indicate the reason for the relationship.
How to Evaluate an Expert's
Statistical Analysis, The Practical Lawyer, April 15, 1982, p. 69.
One of the most common uses of statistics is cases in which a party is alleging discrimination in the composition of the jury panel. The statistics are used to show that the probability of the particular jury panel's composition is so unlikely that discrimination may be present in the process.
Castaneda v. Partida,
430 U.S. 482 (1977) is the leading case on discrimination in the selection
of a grand jury panel. The Supreme Court looked to statistical evidence
to hold that the Texas "key man" system for selecting grand jurors was
unconstitutional under the equal protection clause. A study of the representation
of Mexican-Americans on the grand jury panel over eleven-years demonstrated
that only 39% of grand jury members had been Mexican-American despite the
fact that Mexican-Americans represented 79% of the county population. The
court noted the 40% disparity and a statistical analysis that the probability
a disparity is less than one in 10 to the 140th power in finding that a prima facia case of discrimination had been established.
The Supreme Court looked to statistical evidence in Duren v. Missouri, 439 U.S. 357 (1979) to hold that women were under-repre-sented on the jury venire in violation of the defendant's sixth amendment right to a jury trial. The state law in question exempted wo-men from jury duty if the so requested.
Despite the fact that women represented 54% of the adult population, they consisted of only 15% of weekly venires.
In both of the above Supreme Court cases, the defense presented specific statistical evidence which the government rebutted with broad attack on the assumptions underlying the statistical evidence. Castaneda at 489-93, Duren at 367-69.
When requesting funds for an expert to conduct a study, the defense must present a sub-stantial threshold showing indicating the necessity for such investigation. In McQueen v. Commonwealth, 669 S.W.2d 519 (Ky., 1984), the Court held that it was not error to deny the defense funds to conduct a search to determine proper representation of a cross-section of the community on the jury panel because the defense attorney failed to make a sufficient threshold showing. The court noted that there was "not one shred of evidence...which indicated any irregularity or underrepresentation." Id. at 521.
Likewise, in Ford v. Commonwealth, 665 S.W.2d 304 (Ky. 1983), aff'd, 841 F.2d 677 (6th Cir. 1988), rejected the defense's request for funds for a second statistical investigation. In this case, the defense did receive funding for and conduct an initial study to show discrimination in the drawing of the petit jury. The Court rejected this study because the study used the county census rather than limiting the analysis to the "eligible population" - registered voters and property owners. The Court then rejected the defense's request for funds for a corrected study because the defense failed to convince the judge via a threshold showing.
Additionally, the statute (KRS 31.110) provided for assistance to needy persons charged with a crime states that such person are entitled to be provided with "necessary services...including investigation and other preparation." We do not conceive that employment of statisticians and mathematicians to examine the representation of recognizable groups on jury venires, especially in the absence of specific knowledge of irregularities, to be included in "necessary services." We know of no statute or principle which would authorize expenditures of public funds to conduct a witch hunt. Id. at 308-09.
In a capital case, Randy Haight v. Commonwealth, No. 94-SC-288, tried in Louisville on a change of venue from Garrard County there were irregularities in the way jurors were selected. The case illustrates the benefit of statistical analysis by a defense expert.
In order to obtain 14 jurors (12 who would decide the case plus 2 alternates), the deputy circuit court clerk, on her own and not at the direction of the judge, drew 3 jurors from the 17 left after the exercise of peremptories by the parties, leaving 14 jurors. These 3 were women, 1 of whom was black. The defense objected to the failure to comply with the selection process of RCr 9.36. As a result, the court directed the clerk to draw the jurors out anew by picking 14 out of the 17, leaving 3 to be excused.
Against all odds, when the clerk redrew the 14 names from the 17, the statistically highly improbable occurred. The clerk explained it at a hearing which occurred 10 days later, "it just so happened that the same three popped up." The hearing further revealed that the clerk did not use balls or cards. She used 8-1/2 inch x 3 inch sheets of papers with the jurors' names on them. There was a marked physical difference between the two sets of juror sheets. The 3 drawn out were folded. The 14 were not folded "enough to make any difference." As the clerk stated, "every one of them except for the 14, they were all folded and I straightened them all back out."
It is statistically highly improbable for the same 3 jurors to be randomly drawn out twice using different selection methods. Rather than a random process, the clerk drew the same jurors as she had done before as a result of failing to remix the cards, or due to using slips which had 3 folded and 14 not folded, or putting the 3 names on the bottom of the stack of names, or some other human defect. The physical conditions were not conducive to random-ness, and made a random process extremely improbable. While the clerk recalled 10 days after the event that she shuffled the cards before the second draw, it is possible she did not do that in light of her drawing the same 3. The clerk's use of strips of paper of varying sizes, 3 of which were folded, was not a "...neutral selection mechanism to generate a jury...." Holland v. Illinois, 493 U.S. 474 (1990). A lottery conducted in this manner would not have sufficient public confidence to be viable.
Kentucky State University's Professor Bruce J. Rose, Ph.D. did an extensive analysis of the probabilities of the above two draws occurring by chance. His statistical analysis showed that there are only 15 chances out of 10,000 tries that the second draw of 14 from 17 would not have selected the 3 jurors. Stated conversely, there is a 99.85% chance that the 3 names would have been pulled on the redraw! This statistical analysis shows that failure to draw any of the 3 names on the redraw did not hap-pen by chance or randomly. "The mind of justice, not merely its eyes, would have to be blind to attribute such an occurrence to mere fortuity." (Frankfurter, concurring).
In Avery, 60 names were drawn from a box containing names of prospective jurors. The white prospective jurors' names were printed on white tickets and the black prospective jurors names were printed on yellow tickets. Of the 60 drawn, none were black. The judge who selected the names "testified that he did not, nor had he ever, practiced discrimination in any way...." Id. at 561. "Obviously that practice makes it easier for those to discriminate who are of a mind to discriminate." Id. at 562. Avery held that a prima facie case of discrimination was established and the state had the burden to disprove it. The "opportunity for working of a discriminatory system exists whenever the mechanism for jury selection has a component part, such as the slips here, that differentiates between white and colored; such a mechanism certainly cannot be countenanced when a discriminatory result is reached." Id. at 564. The slips in Haight's case were differentiated with 3 being folded, 14 unfolded, and the names of the 14 readable. The statistical analysis in this case placed a dramatically precise calculation on the improbability of eliminating the same 3 people in two draws.
Other Statistical Issues
Other areas to consider presenting statistical evidence include: jury selection, change of venue, identification (DNA, fingerprints, bloodtyping, etc...), community standards in obscenity cases, media contact, and sentencing.
Art: The Danger of Receiving Statistical Evidence without Effective Rebuttal
The use of statistics presents the danger that mindnumbing numbers will be erroneously produced and relied on by fact-finders. In Chumbler v. Commonwealth, 905 S.W.2d 488 (Ky. 1995). The prosecutor attempted to link a cigarette butt found at the scene to the defendant. The butt was tested and found to be that of a type A secretor, the same type as the defendant. The brand of the cigarette was narrowed to twelve possible brands one of which the defendant smoked. In closing argument, the prosecutor multiplied the percentage of the population who are type A secretors by the percent-age of Americans who smoke by the nation-wide share of the type the defendant used to come to the odds 2 in 1000. He misexplained to the jurors, "And that's the odds that [the defendant] was standing there, 2 in 1000... would've flipped down a Newport cigarette." Id. at 495.
The Kentucky Supreme Court found these calculations "completely unfounded and in error" and despite the lack of objection found "palpable error affecting [the defendant's] substantial rights resulting in manifest injustice." Id. The Court compared the prosecutor's erroneous statistical calculations to the "polygraph in their unreliability and propensity to mislead and may have convinced jurors of modest analytical ability that no one but [the defendant] could have committed the crime." Id. Statistics can be unintentionally miscalculated and the mistake can be missed by defense counsel, the judge and the jurors. In cases with more complex calculations the potential for a mistake to pass through the court increases.
Cases in which only one party offers statistical evidence present the danger that the fact-finder will be awed by the incredible numbers and incomprehensible mathematics. This may lead to jurors taking the understandable shortcut of merely believing the conclusion made by the expert. Dubose v. State, 662 So.2d 1156 (Ala. Cr.App. 1993), aff'd, 662 So.2d 1189 (Ala. 1995), illustrates danger of misleading the fact-finder and the importance of providing the defense with expert assistance to counteract this effect. At trial, the prosecution presented statistical evidence on the possibility of the defendant having the a DNA "match" with samples taken from the victim. The state's ex-pert first estimated that there is a 1 in 500 million probability that the defendant's DNA pattern would appear in the North American black population and a 1 in 22 million probability that that pattern would occur in the North American caucasian population. Then he stated that there are between 15 to 20 million African-American males in North America explaining that "what the statistics tell us here is you would only expect to find this pattern once and we have found it." Id. at 1164.
He concluded by stating that his calculations used three standard deviations, thus producing conservative or significantly underestimated frequencies of occurrence that favor the accused and that without the use of standard deviations, the frequency of occurrence (presumably in the black male population) would be approximately 1 in 500 billion. Later on cross-examination, the expert stated his calculations were based on a database of 1,300 people. Id. at 1164-65.
To most jurors, attorneys or judges this testimony appears to extremely convincing that the sample found on the victim is a match with the DNA of the defendant. In this case, this was critical evidence which led to the jury finding the defendant guilty of murder during a kid-napping, murder during a rape, and murder during sodomy. The defendant was then sentenced to death.
Before reading further look again at the testimony by the state's expert in Dubose, supra, for errors, assumptions, and misleading conclusions made to the fact-finder.
Following is a short list of the problems with this expert's testimony:
2. Study Design. The expert limits his exam-ination only to North American black males. What is the probability that this sample could be from another population? If the probability of this sample matching with an hispanic male is 1 in 400 million then the probability of the assailant being hispanic rather than the same race of the defendant is greater.
3. Sample. What is the evidence that this sample size is large enough for this analysis?
4. Data Collection. How was this sample chosen? How is a black person defined for this analysis? Is a person whose great-great grandfather is white considered black?
5. Result. What is the confidence interval? In other words, what is the percentage of error?
6. Result. The expert put the 1 in 500 billion number in front of the fact-finder without explaining importance of using standard deviations to reach a reliable conclusion and why that number should not be used.
1. What formula did the expert use to generate the final number?
2. Has the expert shown that the formula itself is valid?
3. What does each variable in the formula represent?
4. Does the proper use of the statistical technique, such as the binomial distribution, the normal distribution, and regression analysis that the assumptions are satisfied in the instant case?
5. What is the source for every number that the expert inserted in a variable in the formula? Is the source itself trustworthy? Is the source admissible?
6. Did the proponent misuse the final number during the trial? For example, did the proponent characterize a probability of the defendant's mere presence at the crime scene as the likelihood of guilt?
7. Is the statistical testimony so complex and the final number so impressive that the jury may be overwhelmed or confused?
8. Did the witness exceed his or her expertise by attempting to draw an inference on a question of law?
Scientific Evidence 2d, Paul
C. Giannelli and Edward J. Imwinkelried, 1993, pp. 476-77.
In Dubose, supra, the trial defense attorney was denied funds for his own expert. In an ef-fort to render effective assistance, the attorney was forced to attempt to demonstrate the complexity of the issue and the possibility for error of the evidence presented by the state's experts through cross-examination without the aid of a defense expert. Counsel's uneducated questions prompted four or five of the jurors to tell the court that they understood the evidence even if the attorney did not, and the jurors asked that they not have to listen to any more questions. Id. at 1185.
The appellate court recognized the need for a defense expert:
Ultimately, after watching this direct and feeble cross of the expert the jurors may, without truly understanding the evidence, accept the expert's conclusion as the truth on the issue. This allows the prosecution's expert to sup-plant the juror's responsibility as the finder of fact. See Mahan v. Farmers Union Cent. Exch., Inc., 768 P.2d 850, 857 (Mont. 1989) ("statisticians may testify that their statistical test show or do not show patterns...but may not testify to the ultimate conclusion. The jury should be the final arbiter of that issue.").
This danger is at its greatest in a criminal case.
Admittedly, the appellate case law does not strongly support the need for funding for statisticians, but creative trial attorneys are requesting and receiving funds for these experts. Defense attorneys requesting funding for expert assistance in presenting statistical evidence as part of their evidence at trial during direct or in support of a motion prior to trial will find it effective to both explain the general value of statistics in fact finding and the specific need for statistics to support the traditional evidence in this particular case. In situations when an expert is requested for rebuttal of prosecution evidence emphasis should be placed on the role of assumptions in statistics and the complexity in interpreting results and discovering errors in conclusions.
1The standard deviation is a method used by statisticians to predict the fluctuations from the expected value. See Castaneda, supra, at 1281 n.17.
2Basic probability and statistics texts describe what is necessary for random selection: "To be considered fair, one would want all cards or capsules or names to have the same chance of being chosen. That is, one would strive to emulate a mathematical or ideal selection in which the probabilities postulated for the possible outcomes are all equal. Thorough mixing or shuffling and blend selection are both essential." Bernard W. Lindgren, Basic Ideas of Statistics (1975) at 57.
JEFF SHERR, Asst. Director of Education. Email: firstname.lastname@example.org
TIM ARNOLD, Assistant Public Advocate. Email: email@example.com
ED MONAHAN, Deputy Public Advocate. Email: firstname.lastname@example.org
Prof. Lawrence A. Sherr,
University of Kansas
Prof. Steven F. Arnold, Penn. State University