Dialogue Across Difference: Practice, Theory, and Research on Intergroup Dialogue

Appendix B

APPENDIX B Analytic Procedures, Effects Tables Appendix B details the estimation methods used to account for missing data, test for baseline equivalence, examine the impact of intergroup dialogue on the hypothesized outcomes, and test the theoretical model for intergroup dialogue. MIssING DATA As detailed in chapter 2, the project successfully tracked students who were randomly assigned to dialogue, control, and the social science comparison groups. Nearly all (95 percent) of participants completed the posttest administered at the end the term in which dialogue and comparison courses were offered and the study team successfully followed up with 82 percent of the original sample one year after the posttest. While these retention rates are high for longitudinal studies, analyses conducted with missing data can introduce bias into the results. Specifically, if those students who did not take the posttest or the survey one year later were systematically different than their counterparts who did, the analyses can lead to erroneous conclusions if applied to the sample as a whole. To adjust for the potential bias introduced by missing data, we used multiple imputation. Multiple imputation was used to create ten imputed datasets for subsequent analysis, which reduces the potential bias introduced 392 APPENDIX B when estimating the effect of intergroup dialogue using only observed data. Multiple imputation procedures (Rubin 1987) replace each missing value with a set of plausible values (across ten datasets) that represent the uncertainty about the right value to impute. Analyses are then conducted separately on each dataset using standard procedures and results are combined across datasets in ways that appropriately account for between and within imputation variance. Multiple imputation, as a strategy to reduce bias associated with missing data, assumes that the data were missing at random (MAR) such that missing data depend on observed data but not on unobserved data. Multiple imputation corrects for this kind of bias using the available observed data to predict and impute missing values. After these relationships are accounted for, MAR assumes that the patterns of missingness are completely random. In contrast, including only observed data in analyses assumes that missing data are missing completely at random (MCAR) before adjustment, a less tenable assumption. Initial analyses of missing data patterns suggest that a number of measured variables were associated with missingness in a predictable pattern such that multiple measures collected on the pretest (including demographic information ) predicted patterns of missingness on the posttest and at one year followup . In choosing the number of variables to enter in the imputation model, we erred on the side of inclusion. The general recommendation for imputation models is to use every available variable in the imputation model (Little and Raghunathan 2004), including the dependent variables (Little and Rubin 2002; Allison 2009). IMPAcT ANAlysEs To create a multiply-imputed dataset for analyses examining the impact of intergroup dialogue, we included all of the outcome measures at each time point (pretest, posttest, one-year follow-up) in the imputation model, as well as a range of demographic variables (institution, gender, race, topic of dialogue , privileged group status, major, immigration status, religion, year in school, parental education, pre-college exposure to diversity—racial-ethnic composition of neighborhood, high school, and place of worship, religiosity, liberalism, and students’ prior participation in courses or programs that focused on issues related to race, ethnicity or gender). Multiple imputation was conducted using SAS PROC MI; the procedure was performed separately APPENDIX B 393 for the dialogue, wait-list control, and social science comparison groups to create ten imputed datasets and the data files were subsequently combined for analysis. sEM ANAlysEs To create a variance-covariance matrix based on a multiply-imputed dataset for the SEM analyses testing theoretical framework, we included all individual indicator variables included in the SEM model (pedagogical features, communication processes, psychological processes, outcomes) as well as demographic variables (gender, race, topic of dialogue, status, year in school, pre-college exposure to and students’ prior participation in courses or programs that focused on issues related to race-ethnicity or gender). Because the SEM analyses focused on the processes that took place within intergroup dialogue , only students who participated in dialogue were included. Multiple imputation was conducted using SAS PROC MI; however, to maximize reliability and the stability of the variance and covariance of the variables used in the estimated model, missing data were multiply imputed 100 times and a combined dataset was used to calculate a variance-covariance matrix...