The Handbook of Research Synthesis and Meta-Analysis

13. Effect Sizes for Dichotomous Data

237 13 EFFECT SIZES FOR DICHOTOMOUS DATA JOSEPH L. FLEISS JESSE A. BERLIN Columbia University Johnson & Johnson Pharmaceutical Research and Development C O N T E N T S 13.1 Introduction 238 13.2 Difference Between Two Probabilities 239 13.2.1 A Critique of the Difference 239 13.2.2 Inference About the Difference 239 13.2.3 An Example 239 13.3 Ratio of Two Probabilities 239 13.3.1 Rate Ratio 239 13.2 Inferences About the Rate Ratio 240 13.3 Problems with the Rate Ratio 240 13.4 Phi Coefficient 241 13.4.1 Inference About
241 13.4.2 Variance Estimation for
242 13.4.3 Problems with the Phi Coefficient 242 13.5 Odds Ratio 243 13.5.1 A Single Fourfold Table 244 13.5.2 An Alternative Analysis: (O
E)/V 244 13.5.3 Inference in the Presence of Covariates 245 13.5.3.1 Regression Analysis 245 13.5.3.2 Mantel-Haenszel Estimator 246 13.5.3.3 Combining Log Odds Ratios 246 13.5.3.4 Exact Stratified Method 247 13.5.3.5 Comparison of Methods 247 13.5.3.6 Control by Matching 247 13.5.4 Reasons for Analyzing the Odds Ratio 248 13.5.5 Conversion to Other Measures 250 238 STATISTICALLY DESCRIBING STUDY OUTCOMES 13.1 INTRODUCTION In many studies measurements are made on binary (dichotomous ) rather than numerical scales. Examples include studies of attitudes or opinions (the two categories for the response variable being agree or disagree with some statement), case-control studies in epidemiology (the two categories being exposed or not exposed to some hypothesized risk factor), and intervention studies (the two categories being improved or unimproved or, in studies of medical interventions, experiencing a negative event or not). In this chapter we present and analyze four popular measures of association or effect appropriate for categorical data: the difference between two probabilities, the ratio of two probabilities, the phi coefficient, and the odds ratio. Considerations in choosing among the various measures are presented, with an emphasis on the context of research synthesis. The odds ratio is shown to be the measure of choice according to several statistical criteria, and the major portion of the discussion is devoted to methods for making inferences about this measure under a variety of study designs. The chapter wraps up by discussing converting the odds ratio to other measures that have more straightforward substantive interpretations. Consider a study in which two groups are compared with respect to the frequency of a binary characteristic. Let 1 and 2 denote the probabilities in the two underlying populations of a subject being classified into one of the two categories, and let P1 and P2 denote the two sample proportions based on samples of sizes n1 and n2. Three of the four parameters to be studied in this chapter are the simple difference between the two probabilities, 1
2; (13.1) the ratio of the two probabilities, or the rate ratio (referred to as the risk ratio or relative risk in the health sciences), RR1 /2; (13.2) and the odds ratio, (1/(1
1))/(2/(1
2)). (13.3) The fourth measure,
(the phi coefficient), is defined later. The parameters  and
are such that the value 0 indicates no association or no difference. For the other two parameters, RR and , the logarithm of the measure is typically analyzed to overcome the awkward features that the value 1 indicates no association or no difference, and that the finite interval from 0 to 1 is available for indexing negative association, but the infinite interval from 1 on up is available for indexing positive association. With the parameters transformed so that the value 0 separates negative from positive association, we present for each the formula for its maximum likelihood estimator, say L, and for its non-null standard error, say SE. (A non-null standard error is one in which no restrictions are imposed on the parameters, in particular, no restrictions that are associated with the null hypothesis.) We assume throughout that the sample sizes within each individual study are large enough for classical largesample methods assuming normality to be valid. The quantities L, SE and w1/SE2 (13.4) are then adequate for making the following important inferences about the parameter being analyzed within each individual study, and for pooling a given study’s results with the results of others. Let C denote the value cutting off...