Commentary--Evaluating Community-Based Health Initiatives: Identifying the Characteristics of Successful Initiatives and Evaluations

Leo Tolstoy begins Anna Karenina with the memorable lines: "All happy families are happy in the same way. All miserable families are miserable in their own way." In their insightful article on community-based partnerships, Shortell and colleagues demonstrate that Tolstoy's perception is equally valid when applied to the many initiatives "to improve access, quality, and outcomes of health care." In short, success requires getting everything right and looks remarkably similar in different cases. Failure can result from any number of deficiencies or problems; consequently, the unsuccessful partnerships all look different. This article provides insights for those currently participating in community health care partnerships as well as for those funding organizations that evaluate these partnerships on how to improve the chances of creating a successful program and successfully evaluating such community-based health programs.

Shortell and colleagues describe the process of evaluating a public- private demonstration project of community health care partnerships by presenting a case study of the Community Care Network (CCN), a multiprogram initiative funded by a number of private foundations. The authors acknowledge the importance of two interconnected ideas: many things other than health care services influence health outcomes and, therefore, intersectoral, especially community-based initiatives are likely to have an important impact on health outcomes. The authors point out, however, that this intersectoral aspect makes community-based programs particularly difficult to administer and evaluate. Shortell and colleagues describe the insights they have gleaned from evaluating these initiatives, especially the need for a strong interrelationship between a successful community health initiative and the process of evaluating that success. While evaluation is a separate activity, it is intimately linked to how an initiative is conceived, designed, and implemented. A clear vision and defined goals are essential to both a successful initiative and the ability to rigorously evaluate its impact. This insight is emphasized in three key findings.

Shortell and colleagues begin by delineating the four dimensions of the CCN vision: community health focus, seamless continuum of care, managing within limited resources, and community accountability. They then specify fourteen different management characteristics associated with these dimensions and conclude that of these fourteen, "six distinguishing characteristics" that are present in the top five sites but absent in the bottom five sites of the Community Care Network are fundamental to success: (1) managing partnership size and diversity, (2) attracting leadership of partners, (3) maintaining focus on the goal, (4) managing conflict, (5) facilitating transitions between different stages of organizational development, and (6) transferring resources and people in response to changing needs.

To many, this may seem like an excessive profusion of dimensions, categories, and characteristics that can be dismissed as the musings of overimaginative and underemployed academics. However, it actually reflects the careful conceptual mapping of the various components necessary to pull off a community partnership in order to elucidate the paths to success and the various pitfalls that can undermine a community health partnership. Shortell and colleagues also indicate the miserable consequences of failure in each dimension of the vision, such as little communitywide impact without a community health focus. Such a detailed "landscape" is a critical guide for people who are going to undertake these initiatives because it identifies the various components that must be considered and addressed. This landscape means that each group does not have to experience the same challenges and rediscover the same set of solutions for success. It also helps funders in identifying the precise places at which data should be collected and on which evaluations should focus.

In addition, Shortell and colleagues emphasize the critical role of community involvement in the success of these partnerships. Frequently, community partnerships are projects done to the community rather than with it. The authors emphasize that "without active input for...