Disaggregating International Regimes: A New Approach to Evaluation and Comparison

4. Cognitional Effectiveness

Cognitional Effectiveness 4 The cognitional problem in focus for international regimes is to build a shared, well-founded understanding of how best to achieve the social purpose of the regime. In fisheries management, that means generating research-based advice that differentiates accurately among alternative management programs in terms of the impacts on the state of targeted and related stocks. Some of the factors that have improved cognitional problem solving in the Barents Sea case are regime-driven and include deepening collaboration among Norwegian and Russian research institutions and the stepwise incorporation of ecosystem modeling in the basis for scientific advice. By treating the quality of scientific advice as something that an international regime may explain, at least in part, this chapter differs from most other contributions on the interface of science and politics in resource management. The focus has usually been on the converse causal connection: how aspects of research activities and scientific advice influence decision making. This chapter applies the disaggregate approach to international regime effectiveness developed in chapter 2. Central here is the Oslo-Potsdam formula for measuring effectiveness as the ratio of actual improvement to potential improvement. Thus, we begin by specifying a scale for measuring actual cognitional problem solving based on two dimensions of scientific advice: its salience, or practical usefulness to managers, and its accuracy in forecasting the impacts of various levels of harvesting pressure . Every year, the scientific advisory body provides a menu of harvesting options, each including a forecast on the future level of the spawning stock that will emerge as a result. The accuracy of those forecasts can be measured because the advisory body also publishes retrospective spawning stock assessments; the most recent assessment is the most reliable , because time-series data are available for more years. This procedure helps specify concretely and validly what would qualify as full 110 Chapter 4 problem solving, as well as reasonable thresholds for the three lower scores for describing the outcome of interest: substantial, modest, or insignificant levels of cognitional problem solving. Besides a conception of full problem solving that allows comparison with actual achievements, the effectiveness yardstick used here needs an estimate of the most plausible counterfactual level of problem solving. As explained in chapter 2, determining whether an outcome would be markedly inferior if there were no regime requires a good account of actual problem solving. That is why the subsequent section develops a specific model of cognitional problem solving based on earlier findings in the study of regime effectiveness. This model is then tested with relevant empirical evidence from twenty-five years of scientific advice on the stock in question. Use of the qualitative comparative analysis (QCA) technique helps pinpoint certain ideal-type combinations of causal properties that reliably deliver either cognitional success or cognitional failure. Those findings about causal combinations reliably delivering either success or failure among actual cases prove very useful in substantiating the most plausible level of problem solving that would pertain if the regime did not exist. The disaggregate approach tackles that large counterfactual question by first answering one that is much smaller and more tractable, namely, what scores the case would achieve on the modeled factors if there were no regime. In the section assessing effectiveness, I examine how well those counterfactual cases fit the success and failure paths. Such a fit is measurable by the set-theoretic rules of intersection and negation and helps to estimate the most plausible outcome, since counterfactual cases should behave in the same way as actual cases do. Actual cases reliably achieve problem-solving scores equal to or higher than their scores on a success path; they likewise achieve failure scores equal to or higher than their scores on a failure path. Accordingly, a good fit with a reliable success or failure path helps to derive lower or upper bounds on the plausible range of counterfactual problem-solving estimates , the last parameter needed for assessing regime effectiveness by the Oslo-Potsdam yardstick. The Problem: Generating Salient and Accurate Advice Building a shared and empirically valid understanding among the participants of a regime on how its social purpose can best be achieved is particularly important in sectors involving resource and environmental management, and states typically create or use separate scientific bodies Cognitional Effectiveness 111 to provide relevant input. Such input varies in salience (Parson 2003; Clark, Mitchell, and Cash 2006, 15), that is, whether it differentiates among the policy options...