In lieu of an abstract, here is a brief excerpt of the content:

279 15 ANALYZING EFFECT SIZES: FIXED-EFFECTS MODELS SPYROS KONSTANTOPOULOS LARRY V. HEDGES Boston College Northwestern University C O N T E N T S 15.1 Introduction 280 15.2 Analysis of Variance for Effect Sizes 280 15.2.1 The One-Factor Model 280 15.2.1.1 Notation 281 15.2.1.2 Means 281 15.2.1.3 Tests of Heterogeneity 282 15.2.1.4 Computing the One-Factor Analysis 282 15.2.1.5 Standard Errors of Means 285 15.2.1.6 Tests and Confidence Intervals 285 15.2.1.7 Comparisons or Contrasts Among Mean Effects 286 15.2.2 Multifactor Models 288 15.2.2.1 Computation of Omnibus Test Statistics 288 15.2.2.2 Multifactor Models with Three or More Factors 288 15.3 Multiple Regression Analysis for Effect Sizes 289 15.3.1 Models and Notation 289 15.3.2 Estimation and Significance Tests 289 15.3.3 Omnibus Tests 290 15.3.4 Collinearity 291 15.4 Quantifying Explained Variance 292 15.5 Conclusion 293 15.6 References 293 280 STATISTICALLY COMBINING EFFECT SIZES 15.1 INTRODUCTION A central question in research synthesis is whether methodological , contextual, or substantive differences in research studies are related to variation in effect-size parameters . Both fixed- and random-effects statistical methods are available for studying the variation in effects. The choice of which to use is sometimes a contentious issue in both meta-analysis as well as primary analysis of data. The choice of statistical procedures should primarily be determined by the kinds of inference the synthesist wishes to make. Two different inference models are available, sometimes called conditional and unconditional inference (see Hedges and Vevea 1998). The conditional model attempts to make inference about the relation between covariates and the effect-size parameters in the studies that are observed . In contrast, the unconditional model attempts to make inferences about the relation between covariates and the effect-size parameters in the population of studies from which the observed studies are a representative sample . Fixed-effects statistical procedures are well suited to drawing conditional inferences about the observed studies (see, for example, Hedges and Vevea 1998). Random- or mixed-effects statistical procedures are well suited to drawing unconditional inferences (inferences about the population of studies from which the observed studies are a sample. Fixed-effects statistical procedures may also be a reasonable choice when the number of studies is too small to support the effective use of mixed- or randomeffects models. In this chapter, we present two general classes of fixedeffects models. One class is appropriate when the independent (study characteristic) variables are categorical. This class is analogous to the analysis of variance, but is adapted to the special characteristics of effect-size estimates . The second class is appropriate for either discrete or continuous independent (study characteristic) variables and therefore technically includes the first class as a special case. This second class is analogous to multiple regression analysis for effect sizes. In both cases, we describe the models along with procedures for estimation and hypothesis testing. Although some formulas for hand computation are given, we stress computation using widely available packaged computer programs. Tests of goodness of fit are given for each fixed-effect model. They test the notion that there is no more variability in the observed effect sizes than would be expected if all (100 percent) of the variation in effect size parameters is explained by the data analysis model. These tests can be conceived as tests of model specification. If a fixedeffects model explains all of the variation in effect-size parameters, the (fixed-effect) model is unquestionably appropriate. Models that are well specified can provide a strong basis for inference about effect sizes in fixedeffects models, but are not essential for inference from them. If differences between studies that lead to differences in effects are not regarded as random (for example , if they are regarded as consequences of purposeful design decisions) then fixed effects methods may be appropriate for the analysis. Similarly fixed-effects analyses are appropriate if the inferences desired are regarded as conditional—applying only to studies like those under examination. 15.2 ANALYSIS OF VARIANCE FOR EFFECT SIZES One of the most common situations in research synthesis arises when the effect sizes can be sorted into independent groups according to one or more characteristics of the studies generating them. The analytic questions are whether the groups’(average) population effect sizes vary and whether the...

Share