Rigorous Morality:Norms, Values, and the Comparative Politics of Human Rights
This article argues that there is a strong role for empirical analysis to be used to address fundamental normative questions. Using human rights as an example, the article shows that the evolution of the international regime of human rights provides a standard against which country level performance can be both judged and explained through the application of empirical approaches in comparative politics. It argues further that different kinds of human rights measures (events, standards, surveys, and official statistics) and comparative methods (large-N, small-N, and single-country studies) offer systematic ways in which to map, to explain, and to understand the variation in human rights abuse around the world. The comparative politics of human rights is a prime example of how the “is” of the world can be used to address the “ought” of international human rights theory, [End Page 1] philosophy, and law. The example of human rights analysis in comparative politics shows a strong role for value-based and problem-based research that remains systematic in its approach while at the same time producing outputs that are of public value.
In late 1989, I was working in the photographic laboratory in the Lauinger Library at Georgetown University when a Jesuit Priest arrived one morning with a roll of film that he wanted us to develop. He asked that we make ten copies of the pictures on the roll. In those predigital days, my boss and I stood in the darkroom shaking the can, drying the film, and then printing the pictures. We were not prepared for what we were about to see. The pictures had been smuggled out of El Salvador and were of the scene at the Pastoral Centre of José Simeón Cañas Central American University (UCA) where six Jesuit priests along with a maid and her daughter were killed.1 The images that confronted us were not what was reported in the mainstream press (i.e., that they were simply shot). Rather, we saw the brutal end results not only of a “simple” set of extrajudicial killings, but the manner of killing has left an indelible mark on me ever since. The official truth commission in El Salvador presented strong evidence that not only had the military given the orders to carry out the murders but also that the officers involved engaged in a cover up.2 The facts of the case and the images that I developed those many years ago illustrate a basic point about the enduring capacity of human beings to do horrible things to one another, which for me, led to the deontological conclusion that what I saw was morally wrong and has galvanized my commitment to a lifetime of human rights research.
In 2000, I found myself in João Pessoa, the Eastern-most tip of South America, for lectures and a field visit to the contested countryside in the state of Paraíba, which is dominated by agriculture and the production of [End Page 2] leather. Our field visit was to the sugar cane fields to the West of João Pessoa to meet with officials of the Comissão Pastoral da Terra (Pastoral Land Commission, CPT) and activists from the Movimento dos Trabalhadores Sem Terra (Landless Workers Movement, MST), where we saw first-hand evidence of the violence associated with the struggle for land. The CPT is less confrontational and works with local landless people to secure access to fallow land on which peasants can farm and participate in the market with the longer-term goal of establishing legal title to the land. The MST is more radical in its approach in occupying land, resisting expulsion, and then cultivating the land with the similar goal for achieving legal title. Both organizations have been the subject of armed violence from private militias hired by landowners.3 Bullet holes in walls, militia towers with visible machine guns, and the human stories of violence provide stark evidence of the reality of the struggle and the ongoing human rights violations that are being committed on a regular basis.4 This particular set of observations provided the basis for a research project on the general relationship between inequality and human rights.5 Brazil generated the hypothesis, which was then tested using large-N quantitative comparative methods that not only revealed a general set of findings about the relationship between inequality and human rights, but also a series of typical cases from different regions in the world where high levels of inequality are associated with high levels of human rights abuse.
I recount these cases not to be overly dramatic, but to illustrate the primary motivation behind my research agenda in the field of human rights. Real-world problems associated with the ongoing struggle for greater protection of human rights have been the mainstay topics in my own research, which has used different kinds of comparative analysis (large-N, small-N, and single case studies)6 to provide systematic empirical explanations for the variation in human rights protection.7 If my work can help explain this variation, then there are policy implications that can have concrete impact on the promotion and protection of human rights.8 My work in comparative [End Page 3] politics is value-oriented, problem-driven, and empirically rigorous. While a research orientation that applies empirical methods to normative questions is not unique,9 it does represent a turn away from any notion that social scientific (or political science) research has to be (or can be) value free, and draws on a deep tradition of empirical work that seeks to make social (and political) science matter.10
This kind of research is in keeping with scholars who advocate for empirical approaches to normative questions. For example, in Justice as Impartiality, political theorist Brian Barry makes the case for an empirical approach in providing systematic assessments of country conditions that approximate notions of justice constructed from a Scanlonian “original position.” He argues that the closer such conditions approximate the original position, “the more reason we have to regard the rules that are established and maintained in it as prima facie just.”11 In other words, empirical analysis of country conditions can and should be done to assess the degree to which an ideal of justice has been achieved; a point which is further developed in Why Social Justice Matters.12 In her American Political Science Presidential Address, “What is Political Science For?” Jane Mansbridge argues that political science is the “only academic discipline specifically organized to study” how best to “govern ourselves.”13 Rather than pitting empirical research [End Page 4] against normative work, these and other examples embody the idea upheld by Max Weber that the choice of problems that are to be studied are always “value relevant,”14 where scientific inquiry and methodological rigor can help address otherwise normative questions.
I argue in this article that contemporary empirical research on human rights in comparative politics uses ideal definitions as delineated in international human rights law as a practical “original position” against which evidence about the state of well-being and treatment of individuals, groups, subnational units, and countries are compared. The relative gap between the “ought” of the international law of human rights and the “is” of country conditions becomes a measurable and comparable object of inquiry for systematic comparative analysis, the results of which can inform larger normative questions concerning the advance of human rights. For many analysts, systematic analysis of human rights violations is an empirical form of “speaking truth to power,”15 where such analysis can challenge institutionalized power, reserve domains of authority, and deeply embedded social and political hierarchies that have underpinned human rights abuses.16
To develop this argument the article is presented in five main sections. Section II shows that despite the absence of agreed philosophical foundations, human rights have been “constructed” through a process of consensus building and legalization that has delineated a set of categories and dimensions that can be operationalized for comparative analysis. Section III uses the case of the murder of the Jesuit priests in El Salvador to illustrate the complexity of human rights events, the challenges for measuring and analyzing “events-based” data in single country studies. It also shows how such studies allow scholars and practitioners to build explanations that can travel to other country contexts.17 Section IV uses the case of Brazil to show [End Page 5] how insights from field observations can generate hypotheses that can be tested through large-N comparative analysis of the kind that has dominated much of empirical political science since the 1980s.18 Section V charts a course at the “meso” level between single-country studies and large-N studies to illustrate the ways in which the comparative analysis of carefully chosen cases can take into account the contextual specificities of different countries, while yielding inferences about the protection of human rights that apply beyond the cases under consideration. The final section argues that the comparative analysis of human rights is grounded in a set of epistemological and philosophical assumptions that transcend the “fact-value” dichotomies in the social sciences in ways that allow empirical methods to challenge the kind of power relations that make the continued violation of human rights and human dignity possible.
II. CONCEPTS AND CONSTRUCTIONS
The “rights tradition” predates what is now articulated as “human rights” and is grounded in what remain largely incompatible and contested philosophical arguments and which variously draw on God, nature, and reason.19 Contestation and skepticism about human rights center on their origins, the claims for their universality, their hegemony as a global moral discourse, their inflation beyond what had been considered a fundamental “core,” and the enforceability of particular categories of rights.20 Alongside these different sets of doubts about human rights, there are other arguments that see them as a modern “construction” or “invention”21 that developed out of the [End Page 6] tradition of citizenship rights, which were then universalized through a set of practices and agreements that have yielded the international system for the promotion and protection of human rights that we now have today. The history of citizenship is one of a struggle for rights, as subjugated populations increasingly articulated their grievances in the language of rights; and as modern states formed, rights became extended through law and enforcement mechanisms that provided greater legal protections to an increasingly wider range of rights concerns.22 The general claim for their universality (which one could argue is merely a claim and not a fact) has been met with an increasing specification of different kinds of human rights and participation of states in the international system for the protection of human rights, which provides one kind of empirical evidence for their increasing appeal (see Table 1). The current system for the promotion and protection of human rights is thus an international version of rights that had long been grounded in the nation state, which is now seen as an inherent feature of all human beings by virtue of their being human.23
Human rights transcend the nation state since they declare individual entitlements to an enjoyment of rights as a universal claim regardless of where an individual resides, while opponents of this view tend to “exaggerate the incommensurability of the ideas prevalent in different societies” and argue that such claims are not viable.24 Human rights are moral claims accorded legal recognition, where states parties to international human rights treaties are obliged to ensure that they respect, protect, and fulfill25 these claims. As expressed in international law, they are “specific norms that emerged from a political project” that commenced in the aftermath of the First World War and gained traction as an immediate consequence of the Second World War.26 The 1945 United Nations Charter endorsed the existence and necessity of human rights, and the international human [End Page 7] rights regime that has since emerged is a “deliberately constructed, partial international order” that consists primarily of states and establishes a set of norms prescribing the behavior of those states that become its members.27 The regime focuses on holding governments accountable for their policies and practices that affect their citizens. The regime is wholly centered on ensuring the protection of human dignity and the prevention of state violations of the human rights regime.
This reading of the historical evolution of human rights combines their philosophical status with their social, political, and legal development, where any final claim to the philosophical origins of human rights remains elusive, but a pragmatic claim to their construction does not.28 The development of human rights through international law establishes a framework and ideal set of conditions that should be obtained worldwide. The challenge for an empirical approach to human rights in comparative politics is to take the overall concept of human rights and to recognize the attempts of the international community to systematize the concept in ways that allow analysts to develop indicators and measures of human rights that capture their complexity and variation.29 Capturing the complexity and the variation of [End Page 8] human rights compliance is vital work needed to assess “actual conditions” within and between countries. To date, there has been significantly more focus in comparative politics on developing comparable measures of human rights that focus on the “respect” (and to a lesser degree) the “protect” dimensions of civil and political rights.30 This narrow focus is a function of the philosophical and ideological preferences within the discipline and the methodological issues associated with different strategies for measuring human rights.31 There has been a strong tendency in the discipline to privilege civil and political rights over economic, social, and cultural rights, which draws on larger normative theorizing around liberal definitions of democracy and the “generations” arguments around the evolution of human rights.32 While the legal advance of human rights attempts to eliminate any hierarchy between them, as they have been traditionally conceived, strong biases in favor of civil and political rights remain in political science.33
In addition to the conceptual biases in the discipline, there is much variation in focus across different attempts to measure human rights. Events-based data projects have focused primarily on counting and estimating extrajudicial killings, disappearances, torture, and arbitrary detention, where the methodological approach to events-based data (see below) has not been seen as particularly suitable for measuring violations of economic and social rights.34 “Standards-based” data have been generated initially to measure a narrow set of civil and political rights, but now include some economic and social rights.35 Survey-based measures capture perceptions and experiences with different kinds of human rights violations and conditions, and have included a broader set of human rights than either events-based data or standards-based data.36 Socioeconomic and administrative data are also useful for measuring all categories and dimensions of human rights, but are [End Page 9] subject to questions of data availability and capacity at the country level for data collection and analysis.37
The focus on specific human rights problems and the associated measures interact with different levels of analysis in comparative politics (see Figure 1 below). Single-country studies tend to use narrative analysis,38 history, process tracing,39 interpretive analysis, and other qualitative methods, as well as events-based data, survey-based data, and socioeconomic and administrative statistics. Small-N comparative studies tend to use narrative analysis, comparative history,40 qualitative comparative analysis,41 alongside standards-based, survey-based data, and socioeconomic and administrative statistics. Large-N comparative studies tend to use standards-based data and socioeconomic and administrative data, where data sets have become increasingly complex to include cross-section (150 ≤; N ≤ 194) and time-series data (20 ≤ T ≤ 40).42 The next section discusses the contours, contributions, and challenges of comparative politics at each of these different levels of analysis. It begins with single-country analysis, moves to large-N global analysis, and then considers small-N analysis.
III. BUILDING FROM THE GROUND UP
The case of the 1989 murder of the Jesuit Priests in El Salvador that opened this article illustrates a range of challenges associated with the systematic analysis of human rights in comparative politics. The details of the case itself emerged from a range of different narratives43 about what was discovered on the morning after they were executed. These multiple narratives need to be captured in ways that can then be used for subsequent analysis.44 Legal [End Page 10] analysis and truth commissions seek to uncover timelines on what transpired, including responsibility for who carried out the killings, as well as who ordered them in the first place. From a social science perspective, there is a larger aim to locate the case in a universe of possible cases,45 to make the determination as to whether the killings were typical and part of a larger trend, or atypical and a rare example of brutality. Social scientific analysis is also dedicated to explaining and understanding the killings in the broader context of processes of rational, structural, and cultural theories;46 revolution, counterrevolution, and civil war;47 ideological and religious confrontation; and the US role in supporting the government of El Salvador at the time. Social scientific analysis attempts to do all this while methodologically the case is framed in terms of being a typical instance, a most likely case, or least likely case (see below).48
Beyond the larger explanatory aim and methodological value, the case itself is a good example of the many challenges facing those seeking to count human rights violations and provide the best statistical evidence for subsequent analysis. The “event” itself was comprised of several “acts” that were committed by several “perpetrators” against several “victims.” These words are in quotations to signify their location in a complex array of roles, acts, and relationships typical of human rights events. There are one or more victims who suffered one or more violations committed by one or more perpetrators. Understanding these “one to many” and “many to one” relationships and the kind of deconstruction of the grammar (i.e., “who did what to whom”) of a human rights event has proved essential in data projects that seek to capture, to aggregate, and to analyze large scale human rights violations committed in single countries.49 Failure to disaggregate events where the individual violation itself is a primary unit of analysis may lead to significant undercounting of violations that have happened, as [End Page 11] well as an overrepresentation of other violations that have been counted.50 The “who did what to whom” model of statistical representation of human rights events data, as it has become known, has made a huge contribution to the work of truth commissions around the world whose mandates have been to provide as complete a picture as possible of past atrocities and large scale violation of human rights.51 The method has also been applied to the estimated number of killings in Syria52 since the advent of the Arab Spring, and in a report on the atrocities of the Habré government in Chad.53 In the case of Peru, the model yielded a set of unexpected inferences about the conflict between 1980 and 2000, which showed that the Shining Path revolutionary group was responsible for 46 percent of the total estimated killings (61,007 ≤ N ≤ 77,552) and the government for 30 percent, while further analysis showed differences in the use of violence from both groups that varied by geography and ethnicity.54
Any systematic comparative analysis of human rights must acknowledge the value and complexity associated with using single country studies for making large inferences. A well selected and well framed single country study may make a valuable contribution to our understanding of why, and under what conditions particular sets of human rights are violated. Any intentional selection of a single country study must also locate the country both in terms of theory and research design.55 A theory of the conditions under which gross human rights violations are more likely to be carried out may be tested through careful case selection. If the expected outcome is observed, then the evidence supports the theory. If the expected outcome is not observed, then the theory is infirmed. The selection of the case is not made, in this instance, on the outcome that is to be explained, but the [End Page 12] explanatory factors that have been identified from the theory.56 A single case is unlikely to prove a theory definitively, even if the number of observations has been raised through the collection and analysis of individual or subnational level data (e.g., events, surveys, socioeconomic, or administrative data). Single country analysis can, however, provide the basis for generating hypotheses (see next section) that are then tested through the analysis of a larger number of countries, confirming and infirming theories, and analyzing “deviant” cases or “outliers” from large-N quantitative analysis.
IV. GOING GLOBAL
Like the case of El Salvador, the case of Brazil comprises a highly complex set of issues relating to human rights. My travels in the Northeast provided me with the opportunity to use Brazil to generate hypotheses about the relationships between different measures of inequality (income and land) and the violation of physical integrity rights. The disparities in land and income distribution in Brazil and the narratives and anecdotal evidence about violence committed against the poorest in Brazilian society (in the urban and rural sectors) led me to derive the following argument:
[A]t a micro level, there are incentives for the “haves” in society to engage in rent-seeking behavior within governmental institutions, to maintain control of their resources, and to exclude access to those resources by the “have nots” in ways that use coercive means that undermine the protection of personal integrity rights. … The distribution, accumulation, and defence of resource allocation at the micro level is historically driven and when aggregated to the macro level suggests that the means for maintaining these patterns of distribution may well include violations of civil liberties and personal integrity rights. Moreover, in the context of many lesser-developed countries, there can be state complicity and even collaboration in acts of coercion. We thus expect to find empirical support for a general relationship between inequality and human rights illustrated through our discussion of the Brazilian case.57
From this basic premise, the analysis built a multivariate empirical model analogous to the burgeoning political science literature that uses large-N quantitative analysis to explain the variation in the protection of human rights. The early studies in this subfield use simpler data sets and concentrate on [End Page 13] domestic sociopolitical variables (regime type, international war, civil war, development, ideology, etc.) that account for the variation in the protection of civil and political rights.58 Subsequent studies use more complex data sets and additional domestic and international explanatory variables, which take into account such issues as foreign direct investment, structural adjustment loans, foreign aid, international treaty ratification, the presence of NGOs, membership of IGOs, and globalization and trade, among other factors.59
In the case of inequality and human rights, the analysis found that even after controlling for other factors, there is indeed a significant relationship between high levels of income and land inequality, on the one hand, and the violation of physical integrity rights, on the other. The relationship is stronger for income than land, where the analysis shows that Brazil is a typical case with high levels of inequality and significant problems with the systematic violation of physical integrity rights. It also shows that Brazil is joined by other typical cases such as Peru and Guatemala in Latin America; Angola, Liberia, Sierra Leone, Somalia, Zambia, and Zimbabwe in Africa; and India, Pakistan, and the Philippines in Asia. This conversation across different levels of analysis in comparative politics, far from being a problem of “separate tables,”60 is a fruitful way to locate specific country cases in larger empirical patterns that can be analyzed statistically.
Beyond this particular example and the methodological leverage gained from looking across levels of analysis, the large-N quantitative analysis of human rights is an exemplar of the empirical approach for which Barry advocates, where the empirical evidence demonstrates patterns of injustice. It uses international human rights standards to derive a discrete set of human rights measures. These measures are then compared across a large sample of countries and time with the purpose of analyzing the degree to which country conditions meet the international standard as well as the degree to which different country conditions explain the deviation from this standard. Large-N quantitative approaches show that “valid comparisons can be made between and among different countries to examine empirically the universal claims for human rights that are made normatively.”61 The identification of [End Page 14] factors that account for the variation in human rights tells us something about what can be done to improve human rights performance. For example, the estimation of marginal effects for variables such as land and income inequality suggests that human rights protection could be improved through policies of redistribution.
While large-N quantitative methods have a number of advantages for providing explanations for the variation in particular sets of rights, they have a number of limitations that make them less appropriate for different sets of rights (i.e., those that are not as susceptible to standards-based scales), less appropriate for different kinds of explanatory variables (e.g., social mobilization, advocacy campaigns, and long historical processes of change), and less able to capture country-level contextual specificities on the lived experience of individuals, shared understandings at the grass roots level of human rights, and the comparative historical evolution in the promotion and protection of human rights. Small-N comparative analysis can thus redress some of the limitations of large-N analysis and enrich explanations and understandings of human rights across countries and time. The greater attention to local context, however, does not mean we need to sacrifice the inferential logic of systematic comparative inquiry.62 Rather, we can harness the logic of inquiry through a careful selection of cases that are to be compared in ways that preserve some of the key analytical strengths of quantitative methods while at the same time remaining attentive to the history and culture of individual countries. Boolean approaches developed by Charles Ragin63 and others try to combine the rich detail of small-N analysis with the rigor of algebra to identify combinations of factors that contribute to outcomes of interest. The work on revolutions for example has fruitfully applied this approach.64
The key challenge in conducting small-N analysis is that all such studies engage in the intentional selection of cases, which can introduce bias and indeterminacy both of which can significantly limit the types of inferences made possible through comparison. In my view, the best methodological publication to date on case selection (which itself draws on very strong [End Page 15] precedents,65 including J.S. Mill66) is the James Mahoney and Gary Goertz piece on what they call “the possibility principle.”67 Previous analytical frameworks on small-N analysis have focused on research design strategies that compare similar outcomes across very different countries (i.e., the “most different systems design,” or MDSD), compare different outcomes across similar countries (i.e., the “most similar” systems design, or MSSD), or some combination of the two. These methods hold constant key variables within countries, while allowing variation in other variables in ways that could be used to support particular empirical explanations. These approaches tend to focus on large macro-variables such as different regime types, social revolutions, social protest, and democratic transitions.68
Mahoney and Goertz, however, show that selection without reflection may produce a series of studies that suffer from indeterminate research designs that despite having compelling findings require analysis of additional cases in order to confirm them. Their “possibility principle” argues that for any research topic (and here I include the analysis of human rights), there is a universe of possible cases to be considered, a smaller selection of relevant cases, and a final selection of cases that feature in the analysis. This final selection is nonrandom. However, it is informed by the research question itself and the theoretical framework of the analysis, which is used to derive a set of what Gary King, Robert Keohane, and Sidney Verba call “observable implications.”69 The combination of the research question and theory is a powerful way to identify a set of possible cases for small-N analysis, including what Mahoney and Goertz call “positive” cases, “negative” cases, “disconfirming” cases, “irrelevant” cases, and those cases that are “impossible” but may well occur.70
This framework is ideal for empirical analysis of the variation in human rights abuse across a small number of countries. For example, human rights abuse can be severe in some countries and time periods (e.g., the authoritarian period in Latin America between the 1960s and 1980s), less severe in other countries and time periods (e.g., postwar Europe and the period of EU enlargement), or under contestation through periods of rapid political change (e.g., the Arab Spring countries since late 2010). In these examples, [End Page 16] country level conditions may be compared to provide explanations for why human rights are (or are not) systematically abused, where the analysis may consider some of the key variables that have not been specified in large-N analysis. Rational, structural, and cultural theories as to why human rights are abused (or not) may narrow the universe of cases to a selection of possible and relevant cases for controlled comparisons in ways that leave degrees of freedom and “property space” for the variables of interest to vary, while keeping the overall number of cases manageable.
There are a number of examples from comparative politics in which a small-N research design has provided new sets of inferences about the conditions under which certain sets of human rights can be more fully realized. My own work with Joe Foweraker compared social movement activity and the protection of citizenship rights, both in principle and in practice, across the four cases of authoritarian Brazil, Chile, Mexico, and Spain to uncover domestic processes of political and legal change.71 We carried out case selection on the basis of identifying liberalizing authoritarian regimes that had an element of social mobilization. In The Power of Human Rights, the authors compared “couplets” of country cases from different regions for the relative progress in the realization of human rights protection as a function of linkages between mobilization from domestic NGOs and support from transnational advocacy networks.72 The comparisons in this study corroborated and extended the findings in our own study through examining the transnational dimension in the struggle for human rights. In Insincere Commitments, Heather Smith-Cannoy compares the sequencing of commitment and compliance with respect to international human rights across the cases of Slovakia, Hungary, Kyrgyzstan and Tajikistan.73 Her selection of cases is based on initial quantitative analysis of treaty ratifications in which Eastern European and Central Asian countries stand out as having high rates of ratification and formal acceptance of those provisions that provide third party oversight of state practices. Her comparisons show how relatively cheap international commitments to human rights actually empowered local organizations to use specific provisions within human rights treaties to bring about progressive change.74
Taken together, comparative methods and their associated measures of human rights, offer rich opportunities for explaining and understanding human rights problems. Figure 1 is a formal delineation of the different options within [End Page 17] comparative research methods for studying human rights. The figure shows first that there is a tradeoff between the scope of countries under comparison (1 ≤ N ≤ 194) and the conceptual “ladder of abstraction,” which refers to the degree to which particular concepts are able to travel across different country contexts.75 Human rights advocates argue that international human rights concepts apply universally, but there is still the need for conceptual “stretching” when applying universal standards to country level practices.
[End Page 18]
Each category of comparative method has associated measures of human rights that are typically deployed, as well as particular kinds of research methods, where more qualitative and interpretative approaches are possible in those studies that feature a smaller number of countries. As the number of countries and complexity of variables increases, there is a natural tendency to push the analysis into the use of quantitative approaches. Even among studies on single countries, the use of events-based data and the “who did what to whom” models involve large and complex data sets that combine multiple sources of data and methods of statistical estimation.
VI. NORMS, VALUES, AND EMPIRICS
This article began with two descriptive portraits of human rights problems, El Salvador in 1989 and Brazil in 2000, as a way to highlight how country conditions motivate empirical research. In the case of human rights, the empirical research, the methods of analysis, and the measures of human rights discussed here are all ways in which human rights scholars and practitioners seek to address larger normative concerns relating to human well-being, the relationship between the state and individual, and broader questions of rights and responsibilities. The idea of using empirical methods to address normative questions is not new, but is one that has fallen out of fashion with the rise of increasingly scientific approaches to studying the political world. I have sought to show, however, that normative theory should map out a set of desirable outcomes and ways of organizing modern societies. Therefore, these conditions may be analyzed through the evidence-inference methodological core of comparative politics. The mapping of methods and measures, however, has shown that it is no simple matter to assess and to compare country conditions in ways that inform our normative theories. Rather, questions of information, source material, conceptual coding schemes, measurement strategies, issues of research design, case selection, and analytical approaches require careful consideration before embarking on the empirical analysis of normative questions. Failure to address these key questions that underpin rigorous empirical analysis may mean that providing explanation and understanding of how to achieve our desirable outcomes remains elusive.
In the field of human rights, the “ought” has itself been hotly contested philosophically, and laterally delineated through advocacy and consensus building through the legalization and codification of internationally recognized human rights.76 Any attempt to enumerate the complete set of human [End Page 19] rights that ought to be protected yields lists that range between forty and sixty rights,77 which complicates efforts to analyze them empirically. The sheer complexity of the killing of the Jesuit priests and their colleagues in El Salvador at this micro-level of analysis shows the challenges associated with this kind of work. In short, empirical analysis of what Barry calls “the actual conditions” in a country is not at all straightforward.78
This challenge of assessing actual conditions affects a wide range of human rights projects, where the application of rigorous methods of analysis can lessen the gap between known and unknown information, and provide the basis for the most reasonable assessment of actual conditions. Far from avoiding value based and problem driven research, human rights scholars and practitioners are now more than ever engaging in empirical analysis of real-world problems. The motivation for this work is progressive. It seeks to explain and to understand the conditions under which violations of human rights take place with a view to preventing their violation in the future. The desirability of certain outcomes is the responsibility of philosophers and lawyers, while the means through which they are obtained is the responsibility of empirical researchers. Only through attention to fundamental questions of methods and measures of the kind outlined here may the field of human rights analysis continue to develop and to make a contribution to the realization of those conditions most conducive to human dignity and human equality. [End Page 20]
Todd Landman is Pro Vice Chancellor Faculty of Social Sciences at the University of Nottingham. Between 1993 and 2015 he was Professor of Government at the University of Essex, and his research and teaching focused on development, democracy, and human rights. He is author of Democracy and Human Rights: The Precarious Triumph of Ideals (Bloomsbury 2013), Protecting Human Rights (Georgetown 2005), Studying Human Rights (Routledge 2006), and Issues and Methods in Comparative Politics (Routledge 2000, 2003, 2008); co-author of Measuring Human Rights (Routledge 2009), Assessing the Quality of Democracy (International IDEA 2008); Governing Latin America (2003), and Citizenship Rights and Social Movements (Oxford 1997, 2000); editor of Human Rights Volumes I–IV (Sage 2009), and co-editor of the Sage Handbook of Comparative Politics (Sage 2009), and Real Social Science: Applied Phronesis (Cambridge 2012). He has numerous articles published in International Studies Quarterly, The British Journal of Political Science, Human Rights Quarterly, Democratization, Political Studies, The Journal of Human Rights, The British Journal of Politics and International Relations, Electoral Studies, Human Rights and Human Welfare, Public Law, and The California Western International Law Journal. He has carried out a large number of international consultancies on the measurement and assessment of democracy and human rights.
1. The victims included Ignacio Ellacuría, Rector of the University; Ignacio Martín-Baró, Vice Rector; Segundo Montes, Director of the Human Rights Institute; Amando López, Joaquin López, and Juan Ramón Moreno (teachers at UCA), and Julia Elba Ramos and her daughter, Celina Mariceth Ramos. See, e.g., The Center for Justice and Accountability, Jesuits Massacre Case: Victims (2014), available at http://www.cja.org/article.php?id=473.
2. United States Institute for Peace, Truth Commission: El Salvador (1993). Recently, successful advocacy has breathed new life into this case along very similar lines as those used in the Pinochet case in 1998 in the United Kingdom (i.e. an appeal to universal jurisdiction) and the Ríos Montt case in 2013 in Guatemala. For background on these cases, see Todd Landman, A Most Unlikely Case: Chile, Pinochet and the Advance of Human Rights, 51 Revista de Ciencia Política 37 (2013); International Crisis Group, Justice on Trial in Guatemala: The Ríos Montt Case, Latin America Report No. 50 (2013), available at http://www.crisisgroup.org/~/media/Files/latin-america/Guatemala/050-justice-on-trial-in-guatemala-the-rios-montt-case.pdf.
3. US State Department, Country Reports on Human Rights Practices, Brazil (2007); Leigh A. Payne, Uncivil Movements: The Armed Right Wing and Democracy in Latin America 104-14 (2000); Human Rights Watch, World Report 2007, at 187 (2007).
4. US State Department, Country Reports on Human Rights Practices for 2012, Brazil (2012); Human Rights Watch, World Report 2013: Brazil (2013).
5. Todd Landman & Marco Larizza, Inequality and Human Rights: Who Controls What, When, and How, 53 Int’l Stud. Q. 715 (2009).
6. Todd Landman, Issues and Methods in Comparative Politics: An Introduction (3d ed. 2008).
7. In previous work, I argued “valid comparisons can be made between and among different countries to examine empirically the universal claims for human rights that are made normatively.” See Todd Landman, Comparative Politics and Human Rights, 24 Hum. Rts. Q. 891 (2002).
8. In addition to teaching and research, I have carried out numerous international consultancy projects on the measurement and assessment of human rights for the United Nations Office of the High Commissioner of Human Rights, the United Nations Development Programme, The International Institute for Democracy and Electoral Assistance (IDEA), The Ministry of Foreign Affairs of the Netherlands, the UK Department for International Development, The UK Foreign and Commonwealth Office, the Canadian International Development Agency, and Minority Rights Group International. These activities have contributed in varying degrees to the advance of human rights as measurement and assessment of human rights conditions has formed part of policy intervention and program development by these different agencies and organizations.
9. There are notable examples of empirical analyses of normative questions. Perhaps the most celebrated example is Aristotle’s normative framework on good and corrupt government, which was derived from an examination of 158 Greek constitutions. The assumptions and development of arguments found in Thomas Hobbes, Leviathan (Richard Tuck ed. 1991) had strong empirical referents in the English Civil War of 1642–1651, while empirical tests of normative theory include such studies as Norman Frohlich and Joe Oppenheimer’s experimental analysis of Rawls and Thomas Pogge’s use of Rawls to analyze global injustice. See generally Sir Ernest Barker, The Politics of Aristotle (1995); Thomas Hobbes, Leviathan (A.P. Martinich & Brian Battiste eds., 2011); Norman Frohlich & Joe A. Oppenheimer, Choosing Justice: An Experimental Approach to Ethical Theory (1992); Thomas Pogge, World Poverty and Human Rights: Cosmopolitan Responsibilities and Reforms (2d ed. 2008); see also Rainer Bauböck, Normative Political Theory and Empirical Research, in Approaches and Methodologies in the Social Sciences: A Pluralist Perspective 40 (Donatella Della Porta & Michael Keating eds., 2008).
10. Bent Flyvbjerg, Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again (2001); Sanford Schram & Brian Caterino, Making Political Science Matter: Debating Knowledge, Research, and Methods (2006); Real Social Science: Applied Phronesis (Bent Flyvbjerg, Todd Landman, & Sanford Schram eds., 2012).
11. See Brian Barry, Justice as Impartiality (1995), quoted in Conceptualising Comparative Politics (Anthony Petros Spanakos & Francisco Panizza eds., 2016).
12. See Brian Barry, Why Social Justice Matters (2005); see also Richard J. Arneson, Does Social Justice Matter? Brian Barry’s Applied Political Philosophy, 117 Ethics 391 (2007).
13. Jane Mansbridge, What Is Political Science For? 12 Persp. Pol. 8 (2014).
14. See, e.g., Fritz Ringer, Max Weber’s Methodology: The Unification of the Cultural and Social Sciences, 122–41 (1997); Lewis A. Coser, Masters of Sociological Thought: Ideas in Historical and Social Context 217–60 (2nd ed. 2003).
15. In a presentation from Patrick Ball, Executive Director of the Human Rights Data Analysis Group, he argues that his work on statistics and human rights is very much a case for speaking truth to power, which is a concrete example of the “is” of a country level human rights situation being used by the “ought” of human rights law to hold those who were in power to account for atrocities committed during their time in political office. His analysis of abuse in Guatemala showed that indigenous people across three municipalities targeted by the military had a risk factor of being killed that was eight times higher than nonindigenous inhabitants of the same municipalities. Showing this kind of disproportionate abuse of human rights across different groups in society statistically maps onto the intentionality required of a legal determination of genocide, as in the case of the 2013 trial of former General Efraín Ríos Montt in Guatemala. See Videotape: Data Mining for Good, available at http://media.ccc.de/browse/congress/2013/30C3_-_5405_-_en_-_saal_g_-_201312291730_-_data_mining_for_good_-_patrick.html.
16. Todd Landman, Phronesis and Narrative Analysis, in Real Social Science, supra note 10, at 27.
17. Patrick Ball, The Salvadoran Human Rights Commission: Data Processing, Data Representation, and Generating Analytical Reports, in Making the Case: Investigating Large Scale Human Rights Violations Using Information Systems and Data Analysis_(Patrick Ball, Herbert F. Spirer, & Louise Spirer eds., 2000).
18. Neil J. Mitchell & James M. McCormick, Economic and Political Explanations of Human Rights Violations, 40 World Politics 476 (1988); Steven C. Poe & C. Neal Tate, Repression of Human Rights to Personal Integrity in the 1980s: A Global Analysis, 88 Am. Pol. Sci. Rev. 853 (1994); Todd Landman, The Political Science of Human Rights, 35 Brit. J. Pol. Sci. 549 (2005); Landman & Larizza, supra note 5; Todd Landman, David Kernohan & Anita Gohdes, Relativizing Human Rights, 11 J. Hum. Rts. 460 (2012).
19. Attracta Ingram, A Political Theory of Rights (1994); Micheline R. Ishay, The History of Human Rights: From Ancient Times to the Globalization Era (2008); Charles R. Beitz, The Idea of Human Rights (2011); The Philosophy of Human Rights: Contemporary Controversies (Gerhard Ernst & Jan-Christoph Heilinger eds., 2012).
20. There is a large literature on the enforceability and justiciability of economic and social rights, where the idea of “progressive realization” remains problematic. See, e.g., Robert E. Robertson, Measuring State Compliance with the Obligation to Devote the “Maximum Available Resources” to Realizing Economic, Social, and Cultural Rights, 16 Hum. Rts. Q. 693 (1994); Michael J. Dennis & David P. Stewart, Justiciability of Economic, Social and Cultural Rights: Should There be an International Complaints Mechanism to Adjudicate the Rights to Food, Water, Housing and Health?, 98 Am. J. Int’l L. 462 (2004); Social Rights Jurisprudence: Emerging Trends in International and Comparative Law (Malcolm Langford ed., 2008).
21. Jack Donnelly, Human Rights, Democracy, and Development, 21 Hum. Rts. Q. 608 (1999); Charles R. Beitz, Human Rights as a Common Concern, 95 Am. Pol. Sci. Rev. 269 (2001); Lynn Hunt, Inventing Human Rights: A History (2008).
22. J.M. Barbalet, Citizenship: Rights, Struggle, and Class Inequality (1988); Joe Foweraker & Todd Landman, Citizenship Rights and Social Movements: A Comparative and Statistical Analysis (1997).
23. See Jack Donnelly, Universal Human Rights in Theory and Practice (1st ed. 1989).
24. Barry, justice as impartiality, supra note 11. Barry’s sentiment also resonates with Alison Renteln’s idea that human rights have “homeomorphic equivalents” in societies that may not use the language of human rights, but may nevertheless share the ideas that underpin them. See Alison Dundes Renteln, International Human Rights: Universalism Versus Relativism (1990); Richard Ashby Wilson & Jon P. Mitchell, Human Rights in Global Perspective: Anthropological Studies of Rights, Claims and Entitlements (2003). See also Stephen Hopgood, The Endtimes of Human Rights (2013).
25. Respecting human rights requires the state to refrain from violating them. Protecting human rights requires the state to prevent the violation of human rights by “third” parties, such as private companies, nongovernmental organizations, paramilitary and insurgency groups, and “uncivil” or undemocratic movements. Fulfilling human rights requires the states to invest in and implement policies for the progressive realization of human rights. See The Universal Declaration of Human Rights: A Common Standard of Achievement (Guðmundur S. Alfreðsson & Asbjørn Eide eds., 1999); Todd Landman & Edzia Carvalho, Measuring Human Rights (2010).
26. James W. Nickel, Making Sense of Human Rights: Philosophical Reflections on the Universal Declaration of Human Rights 7 (2007).
27. Andreas Hasenclever, Peter Mayer, and Volker Rittberger, Integrating Theories of International Regimes, 26 Rev. Int. Stud. 3 (2000).
28. Stephen Hopgood argues that precisely because no agreed moral or philosophical foundations have been established, international human rights activists and advocates (who he sees as largely middle class Europeans) have used law and pragmatism to reify human rights in ways that are akin to Bourdieu’s notion of “social magic.” See Hopgood, supra note 24; Pierre Bourdieu, Language and Symbolic Power (1991).
29. For a framework on how to measure concepts for empirical research in the social sciences, see Robert Adcock & David Collier, Measurement Validity: A Shared Standard for Qualitative and Quantitative Research, 95 Am. Pol. Sci. Rev. 529 (2001); Landman & Carvalho, supra note 25.
30. Landman & Carvalho, supra note 25, at 128.
31. Landman, The Political Science of Human Rights, supra note 18; Todd Landman, Studying Human Rights (2006).
32. Karel Vasak, Human Rights: A Thirty-Year Struggle: The Sustained Efforts to Give Force of Law to the Universal Declaration of Human Rights, 30 UNESCO Courier 29 (1977); Michael Freeman, Human Rights: An Interdisciplinary Approach (2002).
33. Economic Rights: Conceptual, Measurement, and Policy Issues (Shareen Hertel & Lanse Minkler eds., 2007).
34. Audrey R. Chapman, A “Violations Approach” for Monitoring the International Covenant on Economic, Social and Cultural Rights, 18 Hum. Rts. Q. 23 (1996); Audrey R. Chapman, The Status of Efforts to Monitor Economic, Social, and Cultural Rights, in Economic Rights, supra note 33, at 143. See also Landman & Carvalho, supra note 25, at 45–63.
35. Standards-based measures of civil and political rights include the two seven-point scales developed initially by Raymond Gastil and now produced annually by Freedom House (see Freedom in the World, available at http://www.freedomhouse.org); the “political terror scale” also developed initially by Raymond Gastil and now produced by academics based at the University of North Carolina at Asheville (see The Political Terror Scale, available at http://www.politicalterrorscale.org); and the “physical integrity rights index,” and other scales produced by David Cingranelli and David Richards (see CIRI Human Rights Data Project, available at http://www.humanrightsdata.com). See also Landman & Carvalho, supra note 25, at 64–90; Human Rights Atlas, available at http://www.humanrightsatlas.org.
36. Landman & Carvalho, supra note 25.
37. Id. at 107–26; United Nations Human Rights Indicators: A Guide to Measurement and Implementation (2012).
38. See Catherine Kohler Reissman, Narrative Methods for the Human Sciences (2008); Landman, Phonesis and Narrative Analysis, in Real Social Science, supra note 16, at 27.
39. See Alexander L. George & Andrew Bennett, Case Studies and Theory Development in the Social Sciences (2005).
40. See Comparative Historical Analysis in the Social Sciences (James Mahoney & Dietrich Rueschemeyer eds., 2003); Gary Goertz & James Mahoney, A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences (2012).
41. See Charles Ragin, The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies (1987); Charles C. Ragin, Redesigning Social Inquiry: Fuzzy Sets and Beyond (2008).
42. Landman, Studying Human Rights, supra note 31; Landman, Issues and Methods in Comparative Politics, supra note 6, at 51–65, 244–48; Linda Camp Keith, Political Repression: Courts and the Law (2012).
43. Landman, Phronesis and Narrative Analysis, supra note 16.
44. Todd Landman & Anita Gohdes, A Matter of Convenience: Challenges of Non-Random Data in Analyzing Human Rights Violations During Conflicts in Peru and Sierra Leone, in Counting Civilian Casualties: An Introduction to Recording and Estimating Nonmilitary Deaths in Conflicts in Peru and Sierra Leone 77 (Taylor B. Seybolt, Jay D. Aronson & Baruch Fischhoff eds., 2013).
45. James Mahoney & Gary Goertz, The Possibility Principle: Choosing Negative Cases in Comparative Research, 98 Am. Pol. Sci. Rev. 653 (2004).
46. Mark I. Lichbach, Social Theory and Comparative Politics, in Comparative Politics: Rationality, Culture, and Structure 239 (Mark Irving Lichbach & Alan S. Zuckerman eds., 1997).
47. See, e.g., Elisabeth Jean Wood, Insurgent Collective Action and Civil War in El Salvador (2003); Neil J. Mitchell, Agents of Atrocity: Leaders, Followers, and the Violation of Human Rights in Civil War (2004); Charles D. Brockett, Political Movements and Violence in Central America (2005).
48. See Harry Eckstein, Case-Study and Theory in Political Science, in Handbook of Political Science, Vol. 7: Strategies of Inquiry 79 (Fred I. Greenstein & Nelson W. Polsby eds., 1975); John Gerring, What is a Case Study and What is it Good For?, 98 Am. Pol. Sci. Rev. 341 (2004); John Gerring, Case Study Research: Principles and Practice (2007); George & Bennett, supra note 39.
49. Ball, supra note 17; Patrick Ball, Jana Asher, David Sulmont, & Daniel Manrique, American Association for the Advancement of Science, How many Peruvians have Died? An Estimate of the Total Number of Victims Killed or Disappeared in the Armed Internal Conflict Between 1980 and 2000 (2003) [hereinafter How Many Peruvians Have Died].
50. See Landman, Studying Human Rights, supra note 31.
51. For example, truth commissions in South Africa, Guatemala, El Salvador, Peru, Sierra Leone, and East Timor have adopted this method. See Landman, Studying Human Rights, supra note 31; Landman & Carvalho, supra note 25.
52. Megan Price, Jeff Klingner, & Patrick Ball, The Benetech Human Rights Program, Preliminary Statistical Analysis of Documentation of Killings in the Syrian Arab Republic (2013), available at https://hrdag.org/wp-content/uploads/2013/02/Benetech-final-SY-report.pdf; Megan Price, Jeff Klingner, Anas Qtiesh, & Patrick Bell, Human Rights Data Analysis Group, Updated Statistical Analysis of Documentation of Killings in the Syrian Arab Republic (2013), available at https://hrdag.org/wp-content/uploads/2013/06/HRDAG-Updated-SY-report.pdf.
53. Human Rights Watch, Le Plaine de Morts: Le Tchad de Hissène Habré 1982–1990 (2013), available at https://www.hrw.org/sites/default/files/reports/chad1013frwebwcover_0.pdf (English language version available at http://www.hrw.org/sites/default/files/reports/chad1213summary_english.pdf).
54. Previous work had estimated far fewer killings and more of a role for government forces, while the narrative of “two Perus” that emerged from the analysis with respect to coastal versus mountainous regions came as a surprise. See How Many Peruvians Have Died, supra note 49.
55. Gerring, What is a Case Study and What is it Good For?, supra note 48.
56. Selection on the dependent variable is highly problematic in comparative politics and may lead to incorrect inferences being drawn about social and political phenomena of interest. Selection on the independent variables is not problematic. See Barbara Geddes, How the Cases You Choose Affect the Answers You Get: Selection Bias in Comparative Politics, 2 Pol. Analysis 131 (1990); Gary King, Robert O. Keohane, & Sidney Verba, Designing Social Inquiry: Scientific Inference in Qualitative Research (1994).
57. Landman & Larizza, supra note 5, at 719.
58. Mitchell & McCormick, supra note 18; Poe & Tate, supra note 18; Stephen C. Poe, C. Neal Tate & Linda Camp Keith, Repression of the Human Right to Personal Integrity Revisited: A Global Cross-National Study Covering the Years 1976–1993, 43 Int’l Stud. Q. 291 (1999).
59. Landman, The Political Science of Human Rights, supra note 18; Todd Landman, Protecting Human Rights: A Comparative Study (2005); M. Rodwan Abouharb & David Cingranelli, Human Rights and Structural Adjustment (2007); Beth A. Simmons, Mobilizing for Human Rights: International Law in Domestic Politics (2009); Emilie M. Hafner-Burton, Forced to be Good: Why Trade Agreements Boost Human Rights (2009); Heather Smith-Cannoy, Insincere Commitments: Human Rights Treaties, Abusive States, and Citizen Activism (2012).
60. See Gabriel A. Almond, A Discipline Divided: Schools and Sects in Political Science (1990).
61. Landman, Comparative Politics and Human Rights, supra note 7, at 891.
62. Ragin, The Comparative Method, supra note 41; Ragin, Redesigning Social Inquiry, supra note 41.
63. Ragin, The Comparative Method, supra note 41; Gisèle de Meur & Dirk Berg-Schlosser, Comparing Political Systems: Establishing Similarities and Dissimilarities, 26 Eur. J. Pol. Res. 193 (1994); Ragin, Redesigning Social Inquiry, supra note 41, at 154–55.
64. See, e.g., Timothy P. Wickham-Crowley, Guerrillas and Revolution in Latin America: A Comparative Study of Insurgents and Regimes Since 1956 (1993); John Foran, Taking Power: On the Origins of Third World Revolutions (2005).
65. Adam Przeworski & Henry Teune, The Logic of Comparative Social Inquiry (1970); Arend Lijphart, The Comparable-Cases Strategy in Comparative Research, 8 Comp. Pol. Stud. 158 (1975); Theda Skocpol & Margaret Somers, The Uses of Comparative History in Macrosocial Inquiry, 22 Comp. Stud. Soc’y & Hist. 174 (1980); Geddes, supra note 56; Andrew Murray Faure, Some Methodological Problems in Comparative Politics, 6 J. Theoretical Pol. 307 (1994).
66. John Stuart Mill, A System of Logic, Ratiocinative and Inductive: Being a Connected View of the Principles of Evidence and the Methods of Scientific Investigation (8th ed., 1879).
67. Mahoney & Goertz, The Possibility Principle, supra note 45.
68. See Landman, Issues and Methods in Comparative Politics, supra note 6, at 131–215.
69. King, Keohane, & Verba, supra note 56, at 28–29.
70. Mahoney & Goertz, The Possibility Principle, supra note 45.
71. Foweraker & Landman, supra note 22.
72. The Power of Human Rights: International Norms and Domestic Change (Thomas Risse, Stephen C. Ropp, & Kathryn Sikkink eds., 1999); See also The Persistent Power of Human Rights: From Commitment to Compliance (Thomas Risse, Stephen C. Ropp, & Kathryn Sikkink eds., 2013).
73. Smith-Cannoy, supra note 59.
75. Giovanni Sartori, Concept Misformation in Comparative Politics, 64 Am. Pol. Sci. Rev. 1033 (1970); Peter Mair, Comparative Politics: An Overview, in A New Handbook of Political Science 309 (Robert.E. Goodin & Hans-Dieter Klingemann eds., 1996); Landman, Issues and Methods in Comparative Politics, supra note 6, at 25–26.
76. The Legalization of Human Rights: Multidisciplinary Perspectives on Human Rights and Human Rights Law (Saladin Meckled-García & Başak Çali eds., 2006).
77. See, e.g., Landman, Studying Human Rights, supra note 31, at 16; Landman & Carvalho, supra note 25, at 20.
78. Barry, supra note 12, at 195.