pdf Download PDF

Davis B. Bobrow - Knights, Dragons, and the Holy Grail - The Good Society 11:1 The Good Society 11.1 (2002) 26-31

Policy Analysis for the Good Society

Knights, Dragons, and the Holy Grail

Davis B. Bobrow


From the menu offered to contributors to this symposium, I have chosen to reflect on the ways in which policy analysis might or might not make a substantial contribution to a good society if we found ourselves in one, and to achieving that desired state (a constructive transition or good society transition). Subsequent sections comment on the role for policy analysis in a good society, obstacles to contributions to a good society transition, steps to weaken those obstacles, and the imperative to relate good society transition contribution attempts to matters beyond the boundaries of any single nation-state. 1 The impediments to and ameliorations for enhanced contributions to a good society transition will, I suggest, retain relevance for policy analysis in a good society.

The frame provided to me is a "big tent," inclusive notion of policy analysis as work "concerned with guiding public decision making by considering the design of discrete public actions, the institutional structure for policy making, and the process of decision making itself." 2 That conception makes policy analysis as an advisory and missionary (conversion and mobilization) activity one of the world's three oldest professions and one of the most universally present among what Harold Lasswell called the "verbal classes." 3

We have then a notable pantheon of past practitioners (to name a few, from Aristotle and Machiavelli to Marx and Lasswell, from Adam Smith to Roland Mckean to Mancur Olson, from Sun Tzu and Clausewitz to Albert Wohlstetter, from Max Weber to Herbert Simon and James Coleman) whether or not self-labeled as policy analysts. 4 Having such a roster of knights is encouraging, but it also surely is humbling and even rather depressing given the vigorous remaining dragons at large in the space between current society and a good society and the back-and-forth nature of good-society transition we have and continue to experience.

The space allotted to me enforces oversimplification and a lack of desirable nuance. My use of that space reflects my belief that policy analysis is better thought of as a profession than as a discipline, and my biases as an analyst of politics in general and international relations in particular. 5 No claim is made for originality in the observations that follow, only that they are inadequately embodied in the prevailing practice of policy analysis.

The Ongoing Challenges of a Good Society

Suppose we woke up one day to find ourselves in a good society. After what would no doubt be a Rip Van Winkle period of disorientation, should we decide that policy analysis is finished business? If the motive and justification of policy analysis had been to narrow the gap between prevailing society and an aspired-to good society holy grail, perhaps we ought to consign policy analysis to a museum of primitive arts and crafts. With the species loss of bad society dragons, why would there be any need for knights to slay them or drive them away?

To do so would be to make the unwarranted assumption that a good society is a completed, immortal work. That would be pragmatically naive and normatively dubious. A relatively likely good society would not be inhabited only by "good people" or be lodged in a world populated only by other entities that matched a good society profile. Stasis is unlikely and even undesirable in the external natural and human context of a good society, the technologies and productive processes available to it, the demographics of its population, and the information provided to or accessible by its component institutions and individuals. Policies and institutions will face continuing and changing design challenges.

A good society, like any complex system, will require substantial maintenance to avoid degraded performance and will have inadequately anticipated "design flaws." It will then need corrective action (ideally of an anticipatory sort based on timely [End Page 26] warnings). Nor will the existence of a good society bring with it the end of history, politics, or economics. The institutions and individuals (including decision makers) who populate a good society will still draw on the precedents or anchors of interpreted experiences, compete for positions of authority, and face allocation choices and finite resources. Accordingly, there will continue to be issues about the expectations and claims warranted by multiple pasts; the constraints and obligations proper to the pursuit and use of power; and the costs, benefits, and effectiveness of alternative uses of scarce resources and institutional arrangements for their management. Uncertainty will not be eliminated about the causality, sensitivity, and timing relationships central to estimating the impacts of policy programs and institutional arrangements.

For all these reasons, public policy in a good society will still face grand issues of ethics and pragmatics, and the sorts of specific choices about tangibles and intangibles associated with one or another school of pre-good society policy analysts.

Advancing a Good Society Transition:
Impediments and Challenges

A far more complex and socially relevant question involves what changes and emphases in the practice of policy analysis will be conducive, and perhaps necessary, for transition to a good society. In the PEGS forum and elsewhere, there has been a great deal of discussion of what we should and should not be doing ranging from narrow technique to basic epistemology. Familiar contests include: positivists against post-positivists, economic rationalists against argument-constructing institutionalists and process advocates, proponents and critics of reliance on markets or governments or civil society organizations against each other, the more modest (incrementalists, satisficers) against the less modest (system synoptic changers, optimizers), and elitists against participatory populists. Instead of concentrating on those debates, I suggest our thinking about policy analysis contributions to a good society transition should start from three sets of cautionary observations.

Synoptic Approaches Versus Sectarian Blind Spots

First, an ample analytic and societal outcome record shows that none of the positions warrants sole reliance or total rejection. None poses generally irrelevant questions or always-wrong answers. Each has "worked" well on some occasions and led to unwanted or at least unanticipated consequences on others. The interactions between the elements in more than one purist position and debate sooner or later assert themselves with non-trivial impact.

For example, the two-way effects between market, government, and civil society organizations condition the performance of each (see, for example, Simon 2000; Olson 2000). The consequences of economic resource transfers are affected by the institutional arrangements and processes in which those transfers take place, effects that shape subsequent policy relevant economic and political matters. Such practical consequences are well illustrated by even casual inspection of the post-1989 Russian experience, or that with U.S. welfare policy. Analysis of the latter has shown the very different "lessons" recipients have learned for political participation from their experience in two welfare programs with quite different administrative structures and routines (Soss, 1999). Those differences manifested in subsequent democratic political participation can have substantial consequences for the broader conduct of government and (especially in close races) electoral outcomes. Experience with democratization finds the same institutional forms associated positively or negatively with a good society transition in some sociohistorical settings having opposite effects in others (Zakaria, 1997; Dalpino, 2000).

Thus the growth of specialization and the claims and expertise of specialist communities among policy analysts may have the same negative consequences as they do for modern medicine: distorted diagnoses, unanticipated and even dangerous interaction effects, treatment of symptoms rather than causes, and excessive reliance on ceteribus paribus assumptions. If so, a particular policy prescription and treatment protocol may seem to specialists to bring with it progress toward a good society, but also may seem and be anything but that for many directly affected parties and broader audiences. 6 Greater contributions to a good society transition call instead for a substantial fusion of understandings of the interacting factors ("dragons") that must be overcome and which intervene between policy analysis treatments and effects. 7

Reformist Intent Versus Special Interest Capture

Second, the growth in numbers of practitioners and organizational providers of one or another sort of policy analysis may well have been accompanied by erosion of the reformist (close the gap between prevailing conditions and good society conditions) orientation of the interwar Chicago social scientists and their students (see, for example, Lasswell, 1936; Lasswell and Lerner, 1951). While grumpy, it also is realistic to recognize that [End Page 27] several developments have attenuated the links between policy analysis at least in aggregate and a good society/good society transition agenda. The developments that seem relevant are hardly unprecedented in the sociology of professions.

Many of the perspectives and technical skills initially intended to move established institutions closer to good society behaviors have been assimilated into (even captured by) those institutions and diverted to serve their pre-existing agendas. Consider, for example, the fate of the economistic approaches introduced into the U.S. Department of Defense in the early 1960s with a near freeze for the last several decades in military service shares of the defense budget.

Fashions for making decisions and assessing options have made policy analysis a standard part of many an advocate's or opponent's arsenal (for policies or institutional arrangements)—as a tool for selling their wares or discrediting the proposals of others. Demand for policy analysis has indeed grown, but the nature of that demand features purchase of rationalization and justification at least as much as reform. Policy analysis comes to serve advocates of an already chosen "product." The analysis is valued to the extent that it provides information and argumentation that aids and abets manipulation of unconvinced audiences and mobilization of those already well disposed toward the advocates' preference. Consider the rise of the public opinion polling and focus groups, and the impact of mini-industries.

The growth in the market for policy analysis brings familiar features of market incentives and market power to bear on current and future practitioners. Reformist motives may well have become less central in career selection and task assignments. Marketing and customer satisfaction ("repeat business") have become more important to competing university and think-tank based policy analysis vendors who need someone to pay their bills.

These trends, even if there were something equivalent to a Hippocratic Oath, have run well ahead of the evolution of professional self-policing. The possibilities of policy analysts becoming pawns ("hired guns") in efforts to maintain or even increase shortfalls from a good society and impede a good society transition now at least rival those of their being agents for good society and good society transition achievement and bolstering.

At the same time, the growth of specialist communities among policy analysts, and the increased professionalization and social recognition of many of them, pose temptations unconducive to good society or transition contributions. Those temptations include an abundance of opportunities to satisfy status aspirations within the relatively narrow worlds of those parochial communities. There is a somewhat seductive pull towards talking mostly with each other about matters which "outsiders" (policy elites and publics) find remote from their predominant concerns. The result makes policy analysis marginal to rather than central in whatever broader developments take place toward or away from a good society.

Compounding Analytic Challenges

Third, most approaches to policy analysis are confronted by contemporary challenges which pose increased difficulties for understanding and prescription. Some only raise the barrier to substantial good society transition contributions. Others do that and, at least potentially, also offer opportunities for greater contributions if we have the wit to exploit them. Secular trends posing challenges include the growing difficulties of decomposing policy and institutional domains into discrete problem sets, the growing reach in distance of developments held to bear on public policy (ripple effects, contagion, precedent setting), the speeding up and growing publicness of events and situations which generate increasingly immediate demands for satisfaction through public policy and instant verdicts on its performance, the growing equalization of capabilities to mobilize for and mount actions to shape public policy with the corollary increase in the number of actors of influence, and simultaneous but not easily compatible trends in political space and identity of increased internationalization and globalization (integration) and localization (fragmentation). 8

Getting to a Good Society through Policy Analysis: Some Partial Remedies

There is then reason to address the possibility of a decline and not just that of an increase in policy analysis contributions to a good society transition. Even those convinced of inherent, highly restrictive limits on possible contributions ought to agree to the need for steps to counter impediments to feasible contributions. Our knowledge of more fully formed professions and corrective measures considered to increase their societal contributions offers some suggestions. As with other professions, those countermeasures will not suffice to eliminate or even permanently [End Page 28] degrade the developments that threaten societal, public interest contributions. The countermeasures offer modest and basic types of ongoing safeguards against the previously noted threats to public interest contributions.

Education and Socialization

One function of the existing institutions and members of a profession is to select, prepare, and shape future members. Several aspects merit increased attention with respect to policy analysts. One is systematic exposure to a usable past of success and failure, and the factors responsible. That implies fostering awareness of major previously proposed and applied ideas and perspectives, bodies of technique and technology, and the sociological and economic characteristics of policy analysis groupings and processes.

Such an emphasis implies greater attention to "classic works" and indicative "projects," modern and ancient. It recognizes that many fundamental issues about policies, and policy institution and process choices, have come around before and thus have a pertinent past (Neustadt and May, 1986). Properly done, "lessons learned" will include awareness of the possibility of omitted facets (variables, events, and expertise) that were not given their due and what led to or helped avoid those omissions and slights. Particular attention would go to the effects of processes and incentives and disincentives for, on the one hand, omissions, and on the other, more inclusive and integrative professional practice. Such steps might increase the chances that successor generations of policy analysts will be less easily seduced by fads and fashions, have a more inclusive and rigorous sense of best professional practice, and be better prepared to resist recurrent temptations to depart from it.

Accountability and Transparency

Given the proliferation of policy analysis vendors in a highly competitive environment, it may well be helpful to adopt conventions enabling more informed consumption of and choice among policy analysis prescriptions, and even professional self-policing. Otherwise, it seems unduly optimistic to expect that policy analysts will predominantly "speak truth to power," or to wealth, or to widely accepted social preconceptions (Wildavsky, 1979).

Some steps analogous to those being discussed and on occasion imperfectly implemented for other professions should be incorporated into policy analysis. These include: a) disclosure of the financial interests and political affiliations of policy analysts and organizations—"this analysis was sponsored by X, an organization funded by Y, whose members stand to gain from Z;" b) track records of policy and policy institution and process prescriptions and of their providers—"batting averages" and "standings" tables; c) "truth in advertising" about the empirical basis for claims (for example, number and representativeness of cases and samples, duration of the periods examined for effects); and d) explicit warnings—"labels" noting known adverse sideeffects for particular populations, conditions, and negative interactions with other simultaneous policies and institutional and process arrangements. Such steps do not amount to having an elite certification institution whose approval must precede use of a policy analysis product. The suggestions can improve awareness of possible bias and estimates of the confidence warranted and risks associated with particular policy analysis explanations, predictions, and prescriptions.

Professional self-policing poses more of a challenge both of incentives and disincentives and mechanisms that do more good than harm. The challenges are exacerbated by the inchoate nature of policy analysis as a common profession and the diverse and very different communities involved. Nevertheless, some possibilities merit consideration. Professional review bodies might well be formed to consider condemnation for non-compliance with the transparency measures just mentioned, or failure to consider factors in other specialist domains well-established as relevant to the policy analysis question under examination.


Most applications of policy analysis amount to competitive advocacy argumentation. That warrants concern with insuring some minimum level of professional representation of the parties with stakes in a particular matter. 9 It seems insufficient to make representation overwhelmingly dependent on the financial resources to buy it, or the prevailing popularity or career advancing publicity gains from association with one or another interest.

We should consider additional steps to provide the policy analysis equivalent of "public defenders" in a serious way. They might involve more recognition of the policy analysis equivalent to "public interest law" as a career specialization. Selection, preparation, and career advancement would pay more attention to a personal propensity to perform pro bono work, and to diffusion of skills in "policy analysis on the cheap" ("guerrilla policy analysis"). 10 Support for public interest, pro bono policy analysis might even become a criterion for high status among policy analysis organizations. A truly level playing field for policy [End Page 29] analysis contestation probably is utopian, but the field can be made much less uneven than it often is now.

Addressing Good Society Transition-Good World Transition Interactions

Among the challenging developments noted earlier, those of globalization and internationalization perhaps pose the greatest challenges to the practice of policy analysis. Bluntly, attempts to contribute to a parochial good society transition can no longer validly treat good world transition matters as exogenous or fixed. Welfare level causes and effects cut across national jurisdictions. Relevant actors, governmental and nongovernmental are often external and indeed transnational in character so that "Who is us?" puzzles proliferate. Experts in a variety of major issue domains, often not identified as policy analysts, increasingly act on that recognition ("behind the border" economic policy, public health, the environment, mass communications, the "new" and "old" security agendas). Institutional arrangements and policy options of a multinational or "anational" or transnational character multiply.

In that context, the explicit and implicit segmentation of policy analysis by governance levels and within governance levels by national location obscure at least as much as they illuminate. A tempting way to come to terms with good society transition-good world transition interactions is a homogenization premise with its emerging uniformity features. A more realistic and promising basis recognizes the simultaneous presence of differences, often vigorously asserted and defended (Barber, 1995).

The implications of these familiar points for policy analysis with respect to education and socialization, accountability and transparency, and representation are rather obvious, and very substantial and demanding. For the first, even the international-domestic side by side of many "comprehensive" public policy professional schools needs to be replaced by fusion across those lines. For the second, the steps mentioned need international content. For the third, improvements in purely domestic representation will not suffice.

A likely response to the previous reflections may well be to discard them as utopian, jaundiced, or irrelevant to the "central issues" in the active debates about policy analysis mentioned earlier. Yet, in light of their historical antecedents, why should we believe that those debates will come to any widely accepted and definitive conclusion or, if continued further, provide additional value to a good society or a good society transition? It seems at least to be worth trying the suggested re-orientation of our energy. That is, we should internalize the positions in those established debates and get on with directions of professional development that are likely to have currently lacking positive consequences for good society and good society transition contributions.


Davis B. Bobrow is a professor of public and international affairs and political science at the University of Pittsburgh.


Barber, Benjamin R. Jihad vs. McWorld, New York: Times Books, 1995.

Bobrow, Davis B. "Policy, Politics and Analysis: Going Beyond Market and Lawyers" American Journal of Politics 21 (1997): 415-19.

Bobrow, Davis B. and John S. Dryzek. Policy Analysis by Design. Pittsburgh: University of Pittsburgh Press, 1987.

Dalpino, Catherine E. Deferring Democracy: Promoting Openness in Authoritarian Regimes. Washington: Brookings, 2000.

George, Alexander L. "The Case for Multiple Advocacy in Making Foreign Policy." American Political Science Review 66 (1972): 751-85.

Khalilzad, Zalmay M., et.al. The United States and a Rising China: Strategic and Military Implications. Santa Monica: Rand, 1999.

Lasswell, Harold D. Politics: Who Gets What, When and How. New York: Whittlesey House, McGraw-Hill, 1936.

Lasswell, Harold D. and Daniel Lerner. The Policy Sciences: Recent Developments in Scope and Method. Stanford: Stanford University Press, 1951.

Neustadt, Richard E. and Ernest R. May. Thinking in Time: The Uses of History for Decision Makers. New York: The Free Press, 1986.

Olson, Mancur. Power and Prosperity: Outgrowing Communist and Capitalist Dictatorships. New York: Basic Books, 2000.

Quade, E.S. Analysis for Military Decisions. New York: American Elsevier, 1970.

Quade, E.S. Analysis for Public Decisions. New York: North Holland, 1982.

Rosenau, James N. Along the Domestic-Foreign Frontier: Exploring Governance in a Turbulent World. Cambridge: Cambridge University Press, 1997.

Simon, Herbert. "Public Administration in Today's World of Organizations and Markets." PS, 33 (2000): 749-56.

Soss, Joe. "Lessons of Welfare: Policy Design, Political Learning and Political Action." American Political Science Review 93 (1999): 363-80.

Wildavsky, Aaron. Speaking Truth to Power: The Art and Craft of Policy Analysis. Boston: Little, Brown, 1979.

Wohlstetter, Albert. "Theory and Opposed Systems Design," Morton A. Kaplan, ed. New Approaches to International Relations, New York: St. Martin's, 1968.

Zakaria, Fareed. "The Rise of Illiberal Democracy." Foreign Affairs 76 (1997): 2-43.


1. Many of my views were shaped by work with John Dryzek (Bobrow and Dryzek, 1987), but he bears no responsibility for the positions taken here.

2. Brian Cook communication, Jan. 17, 2001.

3. The advisory and missionary roles suggest that policy analysis includes both technocratic skill specialists and symbol specialists.

4. That raises the question of why so much contemporary policy analysis professional education acts as if older work merits little attention.

5. The "profession" perspective and some of its major implications are briefly sketched in Bobrow (1977).

6. "The operation was a success but the patient died" might well apply, for example, to the Dayton Accord vis-à-vis achieving a mutually acceptable multicommunal society in Bosnia.

7. The point has been central for advocates of systems and end-to-end analysis such as E.S. Quade (1970, 1982) and Albert Wohlstetter (1968). It goes beyond adding together suggestions from different specialist sects without full consideration of their interactions—as illustrated by suggestions for a U.S. policy of "congagement" toward China of 1/2 military containment and 1/2 political economy engagement (Khalilzad, et al., 1999).

8. James N. Rosenau's (1997) term "fragmegration" captures this duality well.

9. See for example George (1972), and the "science court" literature. Like the American justice system, a genuine rather than a formal advocacy competition requires at least roughly equivalent professional representation.

10. Such steps could enable fuller advantage to be taken of opportunities for "voice" provided by such devices as public comment periods and public hearings. Otherwise, like unregulated contributions to political campaigns, those opportunities may only amplify inequalities in resources already present among parties whose interests are involved.