Rethinking Policy Analysis:
Citizens, Community, and the Restructuring of Public Services
In the last 25 years, the role of government and the relationship between citizens and their government has undergone dramatic change in the United States. Attacks from both the left and the right on large-scale bureaucracy for its inefficiency, lack of accountability, inequitable service, and deprivation of individual rights, have led to fundamental changes in policy design. Instead of policies delivered from the top down by federal agencies, policies today rely heavily upon lower-level governmental and nongovernmental agencies. A wide variety of policy tools are employed which include contracting with private nonprofit and for-profit organizations, tax credits, vouchers, loans, and loan guarantees, to name just a few.
At the same time, government itself has been "reinvented." Public agencies (and their private contractors) are encouraged to compete with other agencies. Much more emphasis is now placed on "customer" service, being responsive to users of public services, and performance measurement (Behn, 2001; Cohen and Eimecke, 1998; Feldman and Khademian, 2000). Clients and citizens are now encouraged to participate in agency decision making, often with formal decision-making roles on advisory boards, committees and boards of trustees. Public agencies are engaging in partnerships with local nonprofit and for-profit organizations to achieve public policy goals such as reducing the welfare rolls and improving job training systems. Many communities now have a multiplicity of community coalitions and partnerships representing a variety of diverse interests often with overlapping memberships. In line with the emphasis on reinvention, public and nonprofit agencies are encouraged to be innovative and entrepreneurial, taking advantage of new opportunities and devising new solutions to longstanding public problems (Cohen and Eimecke, 1998).
Part of the so-called "new public management" reflects the perception by many scholars and policy-makers that public and nonprofit managers have a lot to learn from the corporate world. The cutting edge texts on business management today emphasize vision, mission, strategy, empowering employees, teamwork, and continuous learning (See Kotter, 1996; Senge, 1990; Katzenbach and Smith, 1999; Collins and Porras, 1997). Indeed, many management scholars argue that effective organizations—whether for-profit, nonprofit, or public—are those organizations that look beyond the financials and include other measures of organizational performance. Moreover, employees throughout the organization are involved in the development of organizational performance measures and monitoring their implementation (Kaplan and Norton, 1996; Collins and Porras, 1997).
Both the new patterns of governance through policy tools such as contracting with private nonprofit and for-profit organizations, and new public management prescribe the building of multiple relationships of communication and trust. Decision-making is more open and participatory, and community and employee involvement is critical to solving problems. High levels of interaction must occur both within and among organizations and the ability to forge links among different networks is fundamental to success. Such relationships draw upon what Robert Putnam calls social capital—i.e., the networks of trust and cooperation in a community (1993a, 1993b). Social capital is both a necessary condition for and a product of new approaches in policy and management. While Putnam's work on social capital has many dimensions, one of his most important points is that the design of public policy can have very important effects on the social capital of a community (Putnam, 1993b). The converse is also crucial: that levels of social capital in a community may affect the success of policy implementation.
Given these developments in policy and management, we will make the following points. First, we need a rethinking of what constitutes good policy analysis. Second, we need to pay much more careful heed to the ways in which institutional design influences the analytic process and the capacity of affected groups to actively participate in the policy process. Third, we need to revamp the teaching of policy analysis so that more varied approaches to the study of policy are taught in our schools of public policy and administration. Fourth, we need to create structures and institutions that allow a range of affected parties and interests access to information and an opportunity for input into the analytic process. [End Page 55]
Thinking Broadly About Policy Analysis
The shift in the structure of public services has not yet been matched by a similar shift in policy analysis and evaluation. Traditionally, policy analysis has served democracy by concentrating on the efficiency and effectiveness with which policy goals were delivered. Using tools from economics, policy analysts have conducted sophisticated means-ends assessments. Political scientists analyzing policy have attended to representation and accountability in policy by examining political feasibility and support, and evaluated the ways in which policies are constructed to garner agreements and the way implementation does or does not take place (deLeon, 1999, Ingram and Smith, 1993, Schneider and Ingram, 1997).
Today, assuming that efficiency, political feasibility, and effective implementation are the only measures policy analysts should apply in measuring policies' contributions to democracy is clearly inadequate. Innovative programs may not be given enough chance to succeed. The contributions of a particular program to the capacity of a community to solve its own problems may be overlooked. The influence of a program in changing the overall system of services in a community may be given insufficient consideration in evaluating the merits of a program. To be sure, effectiveness and efficiency are very valuable measures for the assessment of program performance, but outcomes also need to be conceptualized broadly so that a range of measures is considered.
Part of the challenge is to think outside traditional disciplinary boundaries about both the process of policy analysis and the measures of success. More than ever, pressure exists to maintain accountability for the provision of public services. Yet, we risk a host of undesirable results if we focus on only a narrow set of measures. As Lisbeth Schorr (1997) observes, outcome measures need to be considered within a "broader accountability context" (p. 126). Many policy interventions are very difficult to evaluate: Data are unavailable; competing stakeholders cannot agree on the appropriate measures to judge effectiveness; some results may be very long-term; and the program may have multiple goals. Innovative approaches to evaluating policy within this broader context are already underway. All too often however, the outcomes of particular policy interventions or social programs are judged by only a few select criteria.
Narrowness in approach not only risks misrepresenting the effects of policy interventions (e.g., was the job training program really successful in placing welfare recipients in permanent employment?) but virtually guarantees that the impact of a program on the local community is neglected. For instance, an important ongoing debate among policy-makers and scholars today is the merits of for-profit versus nonprofit service delivery. This issue is particular pressing in health care. One aspect of this debate is whether nonprofit health care organizations (especially hospitals) deserve their tax-exemption, given that no significant differences exist between many nonprofit and for-profit hospitals in terms of the amount of charity care provided. But as noted by Gray and Schlesinger (2000), this focus on charity care—an easily accessible measure—ignores the broader community impact of nonprofit and for-profit hospitals. Nonprofit hospitals may be much more effective in building social capital broadly defined through appointment of community advisory boards, the engagement of volunteers, community outreach programs, and the support of other local institutions.
Institutional Design and Policy Analysis
This health care example underscores the extent to which the structure of organizations, and more broadly the design of policy, may have effects on local communities. The increasing interest in community organizations, social capital and citizen involvement in public services (including the evaluation of public services) requires much greater attention to the linkages between policy and program design and citizens. For instance, the structure of reporting relationships in implementation and service delivery embodied in policy has a profound impact on citizen engagement. The ways in which various publics are treated by policy, whether their views of problems are recognized as legitimate or ignored, whether they are targeted for burdens or benefits, the rules to which they are subjected, such as means testing, and the reception they encounter in interaction with implementing agencies all teach lessons about democracy. Policies send implicit messages about which citizens are regarded by government as important and to be served without question and which are to receive benefits only after proving that they are needy. These different messages may undermine notions of democratic equality and the efficacy of citizen involvement.
Further, policy analysts need to be aware of the impact of the social construction of policy problems on the definition of policy problems and proposed solutions. As articulated by Gusfield [End Page 56] (1982), Schneider and Ingram (1997) and others, public problems are socially constructed through a framing process involving a variety of political phenomena including existing policies. The definitions embodied in policies that characterize what is at stake in particular subject areas can lead to processes of democratic discovery or drastically limit participation and debate. Different problem definitions locate political discourse in particular value contexts where certain concerns, such as scientific fact, override other concerns, such as equity and justice. For example, the inability of scientists to find a definitive link between exposure to toxins and incidence of illness in a certain neighborhood may satisfy government regulators but hardly addresses the question of why certain minority populations in particular areas are subjected to inordinate environmental risk. Framing the location of undesirable land uses in minority neighborhoods as an environmental justice rather than environmental health issue has real consequences for the content of and participation in the debate.
Different problem definitions and value contexts also engage particular kinds of participants, such as scientists rather than grassroots groups, and elicit different institutional responses, such as university research programs rather than petition drives. Thus, according to the way the issue is framed, different boundaries of interest or jurisdiction are created. Different people get involved, for example, when domestic violence is defined as a health issue rather than a criminal justice issue, and different values are at stake when an issue is framed in moral rather than economic terms. Defining water pollution problems as an example of human disregard of the natural world evokes a very different public response than regarding the assimilative capacity of water as one of many economic uses that may or may not have a higher value than recreation.
Framing also affects participants' empathy or willingness to see other perspectives and the likelihood of compromise. An example of how a policy can frame an issue in a way that has adverse effects on reasonable discourse is the Superfund legislation. In an excellent example of policy analysis, Mark Landy (1993) argued that the goal of the act, which insists on cleaning up all toxic and hazardous waste dumps to all applicable standards, does not encourage people to think intelligently about the issue. The policy appears to establish a total freedom from risk, and the policy does not encourage citizens to deliberate over which allocations of scarce dollars for clean-up efforts are most desirable. The lesson is that good policy analysis must consider policy designs and how they affect citizen engagement and community involvement. What is the impact of the creation of specialized districts for particularized service delivery? What are the impacts of the movement away from geographically-based to service-based geographical lines? What are the implications of the growing emphasis on neighborhood associations? How is the Internet affecting the framing of issues and the formation of arenas for discourse?
Analyzing policy designs includes a close examination of implementation structures and the ways in which they involve citizens. Devolution and contracting out are often justified as a strategy to move public services closer to the people. While these designs arguably may bring implementation and service delivery structures closer to local people, their impact upon [End Page 57] democracy and citizenship varies widely. Some policy analysts have studied examples of devolution and find that such policy mechanisms spur citizen mobilization and volunteerism (Gonzales-Baker, 1993; Marsten, 1993). Others find that government funding of nonprofits leads to professionalization of staffs, lowered dependence upon volunteers and community ties, and competition among nonprofits for particular service niches (Smith and Lipsky, 1993). The broad implementation of managed-care arrangements within health and social policy would appear to make the connection between government and citizens even more indirect, reducing greatly the incentive or opportunity for citizens to become actively engaged in policy implementation and oversight. Much would seem to depend upon the particular policy design and the resulting nature of the public-private partnerships within particular contexts. This is a clear area for further analysis.
Contracting, vouchers, and other partnerships are often successful in building public support for services to dependent groups lacking in political power. The contract agency provides a service for government using governmental funds. In the process, the contract agency becomes a client of government with keen interests in perpetuating and raising funding for the program. Providers band together in supportive associations and supporters include board members and staffs of private organizations. Since service providers have roots in the community, local support for programs often rises.
Service learning programs, in which government forgives some part of student loans in exchange for work in publicly oriented agencies, can facilitate civic engagement and support. In the case of Americorps, students repay some of their college tuition while at the same time becoming actively engaged in community problem solving. The evaluation of the impact upon participant attitudes and behaviors is still preliminary and more analysis is need, but there is preliminary evidence that service increases the propensity of participants toward greater participation in nonprofit associations (Simon and Wang, 2000).
In sum, policy analysts need to be attentive to the relationship between policy designs, including the implementation structures, and citizen engagement and input into the policy process. Not only is such engagement often critical to policy success, but it is also central to the building of civic capital and an important component of effective public management.
Educating Policy Analysts
The training of policy analysts has received a great deal of attention in recent years. The curricula of public policy and public administration programs have been revised in response to new thinking on the craft of policy analysis as well as external political and social developments. Consequently, what follows are suggestions on issues to consider as schools continue to evaluate and rethink their approaches to educating policy analysts.
Despite the revision of public policy texts and public policy school curricula, in general, policy and administration schools continue to emphasize quantitative analysis techniques including applied microeconomics and statistics. To a certain extent, this is appropriate: analysts need to be well grounded technically. But the devolution of public policy (with the corresponding growth in importance of state and local government) and increased privatization of public services means that many policy analysts are now evaluating local public, nonprofit and for-profit service programs—or policies relying on a mixed public-private delivery system. These programs include local job training initiatives, community-development projects, and new community-based treatment options for the chronic mentally ill. These programs require an understanding of and appreciation for qualitative research methods including comparative case study design, survey research, focus groups, and participant observation. Knowledge of these research methods is also necessary if policy analysts are to understand the more intangible and difficult-to-measure outcomes of policy interventions, including the program's impact on citizens and the broader community.
Examining how policies and implementation structures frame debates, mobilize citizen involvement, engage citizens, and provide for accountability often calls for qualitative studies. Unfortunately, the teaching of qualitative methods is often on the periphery of the curriculum, with qualitative methods an infrequently taught elective, or it is given too little attention within the required analytic course sequence. While many schools are creatively striving to increase the teaching of qualitative methods within the curriculum, much more needs to be done.
Another needed curriculum change is the greater integration of organizational and political analysis with traditional policy analysis. The new governance structure of multilevel and multisector partnerships requires a deep understanding of political motivations and how and when organizations learn and change. If indeed policy analysts are going to undertake analysis with an appreciation for the broader context, then analysts must also understand political and organizational imperatives and how they affect program implementation and analysis itself.
Policy Analysis, Transparency, and Accountability
Over the years, policy analysis has been sometimes criticized as elitist and serving the establishment in power (Lynn, 1999; deLeon, 1997). This perception has led some scholars to regard policy analysis as a fatally flawed enterprise. We are sympathetic with the position that policy analysis has sometimes been used to support the powerful. But it is equally true that policy analysis can be used as a tool for underrepresented or otherwise disadvantaged groups to achieve political influence and policy change. The challenge for the practice of policy analysis is to improve the access of citizens to information and increase the openness of the policy process.
Greater openness and transparency are particularly important today given the diversification of policy tools and the devolution of public policy. The traditional notions of accountability through politically elected and appointed officials operate poorly in an era of decentralization, and public-private partnerships. Accountability for complex services like health care to elderly citizens is terribly difficult because there are so many governmental and nongovernmental entities involved at various levels without clear lines of authority. Such programs garner wide support because there are so many beneficiaries involved in the delivery chain, each of whom gains resources of various kinds. Moreover, complexity inhibits the availability of clear information about who is benefiting from the program and how. Child [End Page 58] welfare agencies, for example, provide keen support for the programs through which they get funding but have resisted evaluations and performance measures.
Many innovations and experiments are underway to improve accountability in the era of decentralization and public-private partnerships with direct relevance to the craft of policy analysis. For instance, the Community Right to Know Act of 1986 lowers the transaction costs of obtaining information critical to citizen education, mobilization and participation. It would be most helpful to future policy designs for democratic governance to know whether information from industries in communities about the amounts and locations of toxic substances actually spurs citizen protests and helps create a sense of community. Benchmarking is a technique increasingly used to improve the performance of public and nonprofit services. Whether the application of "best practices" criteria to measure performance actually helps accountability is open to question, partly because lessons about "best practices" are frequently based upon only a few cases or (even a single case). Similarly, organizational report cards have been used to provide information to the public in modes that are easily understandable. The extent to which the report cards actually serve to provide accountability is badly in need of analysis (Smith and Ingram, forthcoming).
It is also true that important examples exist of the use of policy analysis by under-represented groups and their allies as a way to change public policy. As just one example, Jonathan Fox (2000) argues that independent evaluations by nongovernmental organizations can be used as social counterweights to change the policies of the World Bank. Indeed, in recent years, the World Bank has become more and more receptive to systematically collected data and analysis that is based on the local knowledge of groups and individuals affected by World Bank programs.
To be sure, greater transparency in the operation of public, nonprofit and for-profit organizations involved in public services is not often easy. And as noted, some policy developments such as managed care are reducing transparency and accountability. Nonetheless, the overall push for greater accountability and more community and citizen involvement in public services and the evaluation of public programs makes it imperative that new structures and institutions be created to make data more widely available. Policy analysis that is itself transparent and attends carefully to the transparency of the policies it evaluates is essential.
Adapting Policy Analysis to the New World of Public Policy and Management
The debate on the future of policy analysis has often been framed as a debate between advocates for traditional policy analysis based upon scientific expertise and a participatory policy analysis which includes citizens actively participating in the analytic process (Steelman and Maguire, 1999; Durning, 1999; Lynn, 1999; deLeon, 1998; Weimer, 1998). While illuminating and fruitful, this debate has tended to overlook the ways in which policy and management have been transformed in the last 30 years. To us, the choice is not between positivist and post-positivist policy analysis. Instead, the issue is whether policy analysis of either kind is adequately addressing the broader issues of community impact and involvement.
We believe that the craft of policy analysis should take lessons from the new public management. As noted by Feldman and Khademian (2000), the work of public managers has been constrained by two false dichotomies: 1) that politics and administration can and should be separated, and 2) that the separation can be maintained by a reliance on objective information rather than subjective information that is presented by the subjects of management decision making. The separation of public policy from issues of politics and organizational behavior, including the framing of issues and shaping of debate, the relationships of organizations to each other and to citizens, and the mechanisms of accountability, leads to narrow-gauged analysis that is not only inadequate but also often misleading. The impact of policy upon democracy, the construction of civic capital and the definition of citizenship must be part of the analysis.
In our view, the restructuring of public services with its emphasis
on a new connection between citizens and government and new modes of
delivery mean that policy analysis needs to creatively tap the views and
inputs of citizens and communities in order to improve public policy
and management. This means recognizing that input from citizens into
the analytic process can often be quite helpful to our understanding of
policy. It also means relying on both objective and subjective information
to assay the impact of policies and to recognize that citizen feedback
and input are increasingly valuable, both in terms of understanding
program effects but also in terms of promoting citizen engagement and
participation in the policy process. In some contexts, this effort may
mean various forms of participatory policy analysis. In other contexts,
it may mean that policy analysts
[End Page 59]
conduct extensive surveys of local communities or policy analysts may
consult with various stakeholders to help frame the problem and the
appropriate analytic approach. By broadening participation, policy
analysts may also be more sensitive to ways in which the framing and
design of policies can affect citizen attitudes and participation. We
recognize that these recommendations for change face a variety of
institutional and resource constraints. We believe that creatively
addressing these constraints is essential if policy analysis is to
successfully adapt to the growing complexity of public policy and the
heightened expectations of the citizenry for increased accountability
for public services.
Helen Ingram is the Warmington Endowed Chair in the School of Social Ecology and a professor in the department of political science at the University of California at Irvine. Steven Rathgeb Smith is an associate professor of public affairs at the Daniel J. Evans School of Public Affairs at the University of Washington.
Baker, Susan Gonzalez. "Immigration Reform: The Empowerment of a New Constituency," Public Policy for Democracy, Helen Ingram and Steven Rathgeb Smith, eds. Washington: Brookings, 1993.
Behn, Robert. Rethinking Democratic Accountability. Washington: Brookings, 2001.
Cohen, Steven and William Eimicke. Tools for Innovators. San Francisco: Jossey-Bass, 1998.
Collins, James C. and Jerry I. Porras. Built to Last: Successful Habits of Visionary Companies. New York: Harper, 1997.
deLeon, Peter. Democracy and the Policy Sciences. Albany: State University of New York Press, 1997.
deLeon, Peter. "Models of Policy Discourse: Insights Versus Prediction," Policy Studies Journal, 26 (1998): 147-161.
Durning, Dan. "The Transition from Traditional to Postpositivist Policy Analysis: The Role for Q-Methodology," Journal of Policy Analysis and Management, 18 (1999): 389-410.
Feldman, Martha S. and Anne M. Khademian. "Public Management: The Forest, The Trees, and Public Value," International Journal of Public Management, 3 (2000).
Fox, Jonathan. "Strategies for Influencing Public Policy: Vertical Integration and Civil Society Policy Monitoring," 2000.
Gray, Bradford H. and Mark Schlesinger. "The Changing and Challenged Nonprofit Sector in Health Care," paper presented at the State of the Nonprofit Sector conference in Washington, Aspen Institute, 2000.
Gusfield, Joseph. Drinking-Driving and the Symbolic Order: The Culture of Public Problems. Chicago: University of Chicago Press, 1981.
Ingram, Helen and Steven Rathgeb Smith, eds. Public Policy for Democracy. Washington: Brookings, 1993.
Kaplan, Robert S. and David P. Norton. "Using the Balanced Scorecard as a Strategic Management System," Harvard Business Review (1996): 75-85.
Katzenbach, Jon. R. and Douglas K. Smith. The Wisdom of Teams. New York: Harper, 1999.
Kotter, John P. Leading Change. Boston: Harvard Business School Press, 1996.
Landy, Marc. "Public Policy and Citizenship," in Public Policy for Democracy, (1993): 19-44.
Lynn, Laurence E., Jr. "A Place at the Table: Policy Analysis, Its Postpositive Critics, and the Future of Practice," Journal of Policy Analysis and Management, 18 (1999): 411-425.
Marston, Sallie A. "Citizen Action Programs and Participatory Politics in Tucson," in Public Policy for Democracy, 119-135.
Osborne, David and Ted Gaebler. Reinventing Government: How the Entrepreneurial Spirit is Transforming the Public Sector. New York: Plume, 1992.
Putnam, Robert D. Making Democracy Work. Princeton, N.J.: Princeton University Press, 1993a.
Putnam, Robert D. "The Prosperous Community: Social Capital and Public Life," The American Prospect, 13 (1993b): 35-42.
Schneider, Anne Larson and Helen Ingram. Policy Design for Democracy. Lawrence: University of Kansas Press, 1997.
Schorr, Lisbeth B. Common Purpose. New York: Anchor, 1990.
Senge, Peter. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday, 1990.
Simon, Christopher A. and Changhua Wang. "Americorps, Social Capital and Institutional Confidence," paper presented at the annual conference of the Western Political Science Association, San Jose, California, 2000.
Smith, Steven Rathgeb and Michael Lipsky. Nonprofits for Hire: The Welfare State in the Age of Contracting. Cambridge, Mass: Harvard University Press, 2000.
Smith, Steven Rathgeb and Helen Ingram. "Policy Tools and Democracy," in The Tools of Government, Lester M. Salamon, ed. New York: Oxford University Press, Forthcoming.
Steelman, Todd A. and Lynn A. Maguire. "Understanding Participant Perspective: Q-Methodology in National Forest Management," Journal of Policy Analysis and Management, 18 (1999): 361-388.
Weimer, David. "Policy Analysis and Evidence: A Craft Perspective," Policy Studies Journal, 26, (1998): 114-128.