Unmarried Couples with Children

11. Mixing Methods: Reliability and Validity Accross Quantitative and Qualitative Measures of Relationship Quality

Chapter 11 Mixing Methods: Reliability and Validity Across Quantitative and Qualitative Measures of Relationship Quality MIMI ENGEL S CHOLARS IN the social sciences have debated the merits of mixed methods research for several decades. Yet, relatively little is known about whether multiple methods provide complementary data or an opportunity for additional insight. In this chapter, I explore two primary questions. First, what value is added to conclusions that can be drawn from mixed methods research as opposed to traditional, strictly qualitative or quantitative designs? And, second, to what extent do qualitative interviews and quantitative survey items that tap similar constructs yield similar results? I use the Fragile Families and Child Wellbeing Study and Time, Love, and Cash among Couples with Children; the latter is a qualitative study of a subsample embedded in the larger quantitative survey.1 This design offers the opportunity to explore the extent to which different methods yield comparable results. This chapter compares measures that tap similar constructs across the TLC3 and the Fragile Family Survey. Because the entire TLC3 qualitative subsample is also part of the Fragile Family Survey sample, individuals’ responses to questions meant to measure similar constructs within each study can be compared. This comparison allows me to explore the following: • How does information provided during in-depth qualitative interviews help to expand or explain brief survey responses? Also, do 255 qualitative methods (in-depth interviews) appear to result in greater disclosure about sensitive subjects than quantitative surveys? • To what extent do responses appear to be reliable across the qualitative and quantitative studies (or, how similar are individuals’ responses across data sources)? • Are there some subjects that people report on more reliably than others? For example, are individuals more reliable across studies when discussing topics that are not stigmatizing (perhaps reporting about affection in the relationship would be easier for respondents than reporting on relationship problems)? • Are there differences in reliability by gender or by educational attainment? • How do similar measures from the Fragile Family Survey and TLC3 relate to outcomes? Specifically, is either data source more or less predictive of whether couples break up? Comparing Data Sources In general, quantitative methods such as those used in the Fragile Family Survey and qualitative methods such as those used for the in-depth interviews of TLC3 provide different types of information. Quantitative research is suited to describing and examining patterns of variation and covariation across a large number of cases, whereas qualitative interviews are more often focused on providing in-depth information (Ragin 1994). The Fragile Family Surveys and TLC3 interviews reflect these different methodological approaches. The Fragile Family Surveys took from twenty to forty-five minutes (typically about thirty) to administer. TLC3 interviews lasted from two to three hours. The Fragile Family Surveys collect brief, factual information about a number of domains including children , relationships, and income. TLC3 interviews were semistructured and often evolved into open-ended conversations. The dialogue for those administering the Fragile Family Surveys was scripted. Interviewers were expected to ask questions verbatim and could not provide encouragement to respondents, as it could be considered to be leading. This resulted in data that were uniform across cases, allowing for the synthesis of information that is often the goal of large-scale survey research. TLC3 interviewers, on the other hand, were trained to word questions and structure conversations to maximize rapport and encourage participants to share a detailed narrative whenever possible. The TLC3 interviews were useful for collecting data that was best shared in narrative form (that is, process data such as an in-depth description of a couple’s relationship history), and the participants often guided the direction of 256 Unmarried Couples with Children the interviews. Other topics that tap people’s emotional experiences such as feelings about partners and children are also captured well using indepth interviews. TLC3 interviews were designed to collect process data that are nuanced and detailed, providing access to a person’s reasoning and interpretation. Interviewers who collected and analyzed data from both TLC3 and the Fragile Family Survey note that the open-ended and participant guided nature of TLC3 interviews resulted in data that were more rich and complex but also more difficult to work with.2 One interviewer who collected data for both studies provided an example, saying that in a Fragile Family Survey, a respondent would simply be asked to provide an answer, on a Likert scale, about the extent to...