Measuring the Success of Local Health Surveys Using Participatory-Based Research Practices

Locally collected information has a compelling relevance to both public and private local institutions, beyond that which national survey findings can provide, powerfully driving policy and advocacy by translating local experiences into real and reliable numbers.^1,2 This is particularly true in the field of public health. Community health surveys are performed for a wide range of purposes—to measure the local burden of health conditions, identify community concerns, gauge awareness of health resources, inform policy, and evaluate public health interventions. A challenge to all community health surveys, irrespective of objective or who conducts them is quality of measurement. Decades of research on survey methods and validation studies have identified how easily gross error or poorly constructed questions can render painstakingly-collected information useless or cause deviation from stated objectives.

In this issue of PCHP, two papers describe the experience of doing local-level community health surveys through academic-community partnerships.^3,4 In each paper, the authors reveal the potential rewards as well as the challenges of such collaborations. Themes common to both papers include the need to provide adequate and flexible time for the iterative collaboration process, recognition that community organizations tend to be mistrustful of academic and government partners due to unequal institutional power, and awareness that community organizations may initially have limited capacity to participate in the survey process or access results. The authors should be commended for their commitment to the participatory research process and for frank assessments of their experiences. What is harder to document is the degree to which such collaborative processes improve salience, measurement, design quality, and utility of survey findings.

The field of community-based participatory research (CBPR) has grown rapidly in recent years. Academic and government institutions can use this inclusive and democratic process to obtain input from communities about local health priorities. Community organizations can play an equal role in guiding research and benefiting from its results. The first and perhaps most important challenge in such collaborations is where institutional agendas intersect, and to establish clearly identified objectives that all parties agree upon. Garcia et al describe the challenge of melding two divergent community objectives into one survey instrument - urban community collaborators, who are mostly service providers seeking to measure knowledge of mental health resources among urban Latino youth, whereas rural community collaborators are local organizers seeking to understand burden of mental health problems among rural Latino adults.³ The decision to address both of these competing agendas undoubtedly added complexity to the survey process and lengthened the development and piloting process to two years. At the time the article was written, the actual survey had not yet gotten underway. In this case, the anticipated trade-off is that well-merged objectives and careful piloting will result in a high quality survey that will make up for the loss in timeliness.

Shirey et al. spend less time in their article describing the respective agendas of partner organizations in the launch of a community health survey.⁴ However, according to the authors; several community partners were initially not supportive of a community survey and suspicious of potential survey results. Perhaps it was the lack of clearly identified objectives, linked to organization agendas, which played a role in the subsequent frustration with the survey development process and nonuse of the data by community partners in 2003. Trust and communication were carefully rebuilt for the second biennial survey in 2005. Nonetheless, the community still did not use the survey results and the authors acknowledge that the partnership might have failed to “include survey items of direct relevance and interest to more of the CBOs”. In this instance, the partnership was timely in conducting the survey every two years - and indeed public health surveillance relies on ongoing and systematic data collection. Assuming the partnership continues, a key challenge for the authors will be to better identify and incorporate the common interests of all partners without compromising survey quality.⁵

In a recent commentary, Farid et al lamented the lack of a standard by which to compare the success of CBPR methods.⁶ For...