In lieu of an abstract, here is a brief excerpt of the content:

497 26 VISUAL AND NARRATIVE INTERPRETATION GEOFFREY D. BORMAN JEFFREY A. GRIGG University of Wisconsin, Madison University of Wisconsin, Madison C O N T E N T S 26.1 Introduction 498 26.2 Displaying Effect Sizes 498 26.2.1 Current Practices 498 26.2.2 Presenting Data in Tables 500 26.2.3 Using Graphs 501 26.2.3.1 Assessing Publication Bias 501 26.2.3.2 Representing Effect-Size Distributions 502 26.2.3.3 Representing Systematic Variation in Effect Size 503 26.2.3.4 The Forest Plot 505 26.2.4 Visual Display Summary 510 26.3 Integrating Meta-Analytic and Narrative Approaches 511 26.3.1 Strengths and Weaknesses 511 26.3.2 Reasons for Combining Meta-Analytic and Narrative Methods 511 26.3.2.1 Describing Results 512 26.3.2.2 Providing Context 513 26.3.2.3 Interpreting the Magnitude of Effect Sizes 515 26.3.3 Integrating Approaches Summary 516 26.4 Conclusion 516 26.5 Notes 517 26.6 References 517 498 REPORTING THE RESULTS 26.1 INTRODUCTION Meta-analyses in the social sciences and education often provide policy makers, practitioners, and researchers with critical new information that summarizes the central quantitative findings from a particular research literature. However , meta-analytic articles and reports are also rather technical pieces of work that can leave many readers without a clear sense of the results and their implications. In this respect, both visual and narrative interpretations of research syntheses are important considerations for presenting and describing complex results in more informative and accessible ways. Typically, readers of meta-analyses in the social sciences and education are offered two types of summary tables to help them make sense of the results. First, a table presents the overall summary effect size for the entire meta-analysis. This table may also include mean effect sizes for subgroups of studies defined by important substantive or methodological characteristics of the included studies (for instance, whether the study employed an experimental or nonexperimental design). Reports and articles often present a second table that includes a lengthy catalog of all studies included in the meta-analysis, along with such information as their publication dates, the authors ’names, sample sizes, effect sizes, and, perhaps, differences among the studies along key substantive or methodological dimensions such as the ages of participants or the nature of the control group employed. Usually the material in these sorts of tables is not arrayed in a particularly informative order but instead simply sorted chronologically by publication year or alphabetically by the authors ’ last names. As we show in this chapter, however, the situation is improving. Such tables can be useful to describe the individual data elements—the outcomes gleaned from each of the studies included in the meta-analysis—and can help present information about the overall effects observed and how those effects might vary by interesting methodological and substantive dimensions. However, we assert that data from meta-analyses can be presented in ways that facilitate understanding of the outcomes and that communicate more information than have been conventionally employed in the literature. Rather than limiting presentation and interpretation of meta-analytic data to simple tables and descriptive text, we argue for greater consideration of figures and displays. This chapter also examines how the careful integration of quantitative and narrative approaches to research synthesis can enhance the interpretation of meta-analyses. We outline three key circumstances under which one should consider applying narrative techniques to quantitative reviews. First, some of the earliest critics of metaanalysis suggested that the methodology applied a mechanistic approach to research synthesis that sacrificed most of the information contributed by the included studies and glossed over salient methodological and substantive differences (Eysenck 1978; Slavin 1984). We believe that many contemporary applications of meta-analysis have overcome these past criticisms through more rigorous analyses of effect-size variability and modeling of the substantive and methodological moderators that might explain variability in the observed outcomes. However, we argue that integration of narrative forms of synthesis can help further clarify and describe a variegated research literature. Second, there may be evidence that some literature , though of general interest and importance, is not amenable to meta-analysis. The typical meta-analysis would simply exclude this literature and focus only on the results that can be summarized using quantitative techniques. We believe, though, that such a situation may call for blending meta-analytic and narrative synthesis so that a more comprehensive body of research...

Share