In lieu of an abstract, here is a brief excerpt of the content:

147 8 IDENTIFYING INTERESTING VARIABLES AND ANALYSIS OPPORTUNITIES MARK W. LIPSEY Vanderbilt University C O N T E N T S 8.1 Introduction 148 8.1.1 Potentially Interesting Variables 148 8.1.2 Study Results 148 8.1.2.1 Results on Multiple Measures 148 8.1.2.2 Results for Subsamples 149 8.1.2.3 Results Measured at Different Times 149 8.1.2.4 The Array of Study Results 150 8.1.3 Study Descriptors 150 8.2 Analysis Opportunities 152 8.2.1 Descriptive Analysis 152 8.2.1.1 Descriptive Information About Study Results 152 8.2.1.2 Study Descriptors 152 8.2.2 Relationships Among Study Descriptors 153 8.2.3 Relationships Between Effect Sizes and Study Descriptors 154 8.2.3.1 Extrinsic Variables 154 8.2.3.2 Method Variables 154 8.2.3.3 Substantive Variables 155 8.2.4 Relationships Among Study Results 155 8.3 Conclusion 157 8.4 References 157 148 CODING THE LITERATURE 8.1 INTRODUCTION Research synthesis relies on information reported in a selection of studies on a topic of interest. The purpose of this chapter is to examine the types of variables that can be coded from those studies and to outline the kinds of relationships that can be examined in the analysis of the resulting data. It identifies a range of analysis opportunities and endeavors to stimulate the synthesist’s thinking about what data might be used in a research synthesis and what sorts of questions might be addressed. 8.1.1 Potentially Interesting Variables Research synthesis revolves around the effect sizes that summarize the main findings of each study of interest. As detailed elsewhere in this volume, effect sizes come in a variety of forms—correlation coefficients, standardized differences between means, odds ratios, and so forth— depending on the nature of the quantitative study results they represent. Study results embodied in effect sizes, therefore, constitute one major type of variable that is sure to be of interest. Research studies, however, have other characteristics that may be of interest in addition to their results. For instance , they might be described in terms of the nature of the research designs and procedures used, the attributes of the subject samples, and various other features of the settings , personnel, activities, and circumstances involved. These characteristics taken together constitute the second major category of variables of potential concern to the research synthesist—study descriptors. 8.1.2 Study Results Whatever the nature of the issues bearing on study results , one of the first challenges the synthesist faces is the likelihood that many of the studies of interest will report multiple results relevant to those issues. If we define a study to mean a set of observations taken on a subject sample on one or more occasions, there are three possible forms of multiple results that may occur. There may be different results for different measures, for different subsamples , and for different times of measurement. All of these variations have implications for conceptualizing, coding, and analyzing effect sizes. 8.1.2.1 Results on Multiple Measures Each study in a synthesis may report results on more than one measure . These results may represent different constructs, that is, different things being measured such as academic achievement, attendance, and attitudes toward school. They may also represent multiple measures of the same construct, for instance achievement measured both by a standardized achievement test and grade point average. A study of predictors of juvenile delinquency thus might report the correlations of one measure each of age, gender , school achievement, and family structure with a central delinquency measure. Each such correlation can be coded as a separate effect size. Similarly, a study of gender differences in aggression might compare males and females on physical aggression measured two different ways, verbal aggression measured three ways, and the aggressive content of fantasies measured one way, yielding six possible effect sizes. Moreover, the various studies eligible for a research synthesis may differ among themselves in the type, number, and mix of measures. The synthesist must decide what categories of constructs and measures to define and what effect sizes to code in each. The basic options are threefold. One approach is to code effect sizes on all measures for all constructs in relatively undifferentiated form. This strategy would yield, in essence, a single global category of study results. For example, the outcome measures used in research on the effectiveness of psychotherapy show little commonality from...

Share