Statistical Methods in Recent Higher Education Research

A broad consensus regarding the need for universities to cultivate academic success has emerged because many students do not achieve their goals or optimally benefit from their higher education (HE) experiences (Altbach, 2014; Davidson & Wilson, 2014; Ferrão & Almeida, 2019; Meggiolaro, Giraldo, & Clerici, 2017; Tinto & Pusser, 2006). This consensus—alongside the expansion of HE in recent decades—is perhaps the main reason behind the growth of HE research (Tight, 2018; Vincent-Lancrin, 2009). Due to the trends expected in HE in the coming years (Organisation for Economic Cooperation and Development, 2008), such growth is expected to continue with increased diversity and complexity (Altbach, 2014), along with stronger institutional and professional competitiveness. The recent debate around academic publishing (Altbach & de Wit, 2018; Lee & Maldonado-Maldonado, 2018) has put increased pressure on academic researchers whose work is in HE, with consequences for their professional development. Altbach (2014) points out that the design of graduate and postgraduate programs plays a crucial role: "The research function of all but a few of the most prestigious programs is less central than training programs" (p. 1310). The consequences on disciplinary advancement may be related to the evidence that, despite massive growth in the number of papers published, the innovative contributions of these studies are limited, and several consist of "repeating analyses that have already been carried out elsewhere in a different context" (Tight, 2018, p. 617).

The primary reason for this study was to identify the statistical methods that were used in HE research between 1998 and 2017. In focusing on these methods, I considered recent trends in the statistical methods used in HE research (Daniel, 2015; Hutchinson & Lovell, 2004; Tight, 2013), their relevance to the development of the field, the need for dialectics between theory and methods, the use of research methods "to interrogate and develop the theories" (Ashwin & Case, 2012, p. 271), and the scarcity of research on methodological characteristics of the published literature (Hutchinson & Lovell, 2004). I evaluated several sources to obtain the selected papers. So far, existing reviews have focused on papers published in 2010 (Tight, 2013, 2014) and published by three leading journals from 1996 to 2000 (Hutchinson & Lovell, 2004) and from 2006 to 2010 (Wells, Kolek, Williams, & Saunders, 2015). Our contribution to the literature is twofold. First, I extend the knowledge in time and in content. Second, I contribute to a deeper understanding of the methodologies used in the field, considering the category of methods, context of development (research of single vs. multiple institutions), geographical origin, and whether such methods are explicitly linked to a theoretical framework. Therefore, this study supports the establishment of strategies that reinforce the field of HE research. The content is relevant to current students pursuing HE research positions, offering them an opportunity to anticipate their training in quantitative methods. It is also relevant for academics responsible for postgraduate programs in education as a resource for curriculum development.

BACKGROUND LITERATURE

Although HE is considered a research field without a clear methodological center (Altbach, 2014; Tight, 2014), recent findings show that quantitative methods continue to dominate (Hutchinson & Lovell, 2004; Tight, 2013; Wells et al., 2015). According to Tight (2013, 2014), who analyzed 567 papers published in 15 leading HE journals in 2010, three of the methods identified—multivariate analyses (including surveys), documentary analyses, and interview-based studies—accounted for 91% of the papers studied; the largest portion (44%) were papers based on multivariate analysis. A detailed analysis of data presented reveals that for papers using multivariate analysis the disciplinary background for most of the first authors (35%) was education, followed by other social sciences (25%). Tight also showed that 39% of the authors with an education background used surveys and multivariate analyses in their articles, 30% used documentary analyses, and 26% used interviews. For those with social science backgrounds, 63% used surveys and multivariate analyses in their articles, 18% used documentary analyses, and 9% used interviews. These figures suggest that multivariate methods were the first choice among HE researchers in general at the time.

Wells et al. (2015) replicated a study conducted by Hutchinson & Lovell (2004) a decade earlier. Content analysis...