Implementing and Evaluating a Counseling Program in Partnership With a Community-based Domestic Violence Service Provider

Abstract

Background: Given the burden of intimate partner violence (IPV), effective counseling interventions that are accessible to women in their own communities are needed.

Objectives: To describe the collaborative process of implement ing and evaluating a new counseling program for IPV—the Women's Initiative for Safety and Health (WISH)—in a community setting and present results of its Thrst pilot test.

Methods: WISH is a stage-tailored, eight-session counseling program based on the transtheoretical model. Imple men tation evaluation addressed program exposure/Thdelity, counselor experience, and client experience. Outcome evaluation measured change in IPV-related incidents, self-efficacy, decisional balance, overall health, quality of life, and stage of change among 19 women.

Results: Both counselors and clients perceived the WISH intervention positively. Across all outcomes, there were improvements among some women, most notably in perceived self-efficacy, decisional balance, and stage of change.

Conclusions: Academic–community partnerships can facilitate the translation of theory-based interventions for use in community-based settings.

IPV is recognized widely as a significant public health problem affecting women in the United States and globally. More than one in three women (35.6%) in the United States have experienced rape, physical violence, and/or stalking by an intimate partner in their lifetime.¹ Numerous community-based organizations provide services for victims of IPV, which can include crisis intervention, shelter, legal services, and short- and long-term counseling. Although these services can be critical for victims and survivors, little information exists on their effectiveness.

Existing evidence suggests that such services have varying levels of success.² Wathen and MacMillian's systematic review concluded that there were no high-quality evaluations of shelter stays, although they found fair evidence that a significant program of advocacy and counseling for women who spent at least one night in a shelter reduced reported violence and improved reported quality of life.³ Other evaluations specific to screening and counseling interventions in health care settings^4–7 have also had mixed results. Sullivan et al.⁸ conducted a review of counseling interventions provided within or in collaboration with domestic violence programs, and found research limited to only six studies, some of which showed promise for reducing depression and anxiety and improving well-being, but many of which had methodological limita tions. Thus, much more needs to be learned about how to effectively address IPV in community-based settings.

Conducting such work through academic–community partnerships could provide new insights into how to best meet survivors' needs. Despite the potential health benefits of academically developed interventions implemented in community settings, successful dissemination and collaboration has often been limited. Barriers to dissemination and implementation of evidenced-based interventions include a lack of organizational resources, overly complex or time-consuming/expensive designs, challenges for community organizations to finding and adapting these interventions, and failure of interventions to address tools and outcomes of interest to community organizations.^9–12 Overcoming these obstacles through solid academic–community partnerships will bolster research on interventions for IPV.

OBJECTIVES

We had an opportunity to address this need through an academic-community partnership between public health researchers (M.V.W., S.I., J.C., A.G.) and practitioners from a domestic violence community service organization (L.M., S.C.S., M.P., M.T.). The aims of this paper are to 1) describe the collaborative process of planning and evaluating a new counseling program, namely, the WISH, and 2) present results of a pilot implementation and outcome evaluation. We conclude with lessons learned regarding both the implementation of the new counseling program in a community service organization, and the academic–community partnership.

METHODS

The WISH Program

WISH is an eight-session, individually tailored counsel ing program that focuses on helping women experiencing IPV stay safe and make informed decisions about the abusive relationship to improve their physical and mental health. WISH is based on the transtheoretical model¹³ and formative research^14,15 that led to deThning five stages of readiness to end the abusive relationship. The five stages are: 1) precontemplation (the woman does not want to end the relationship or is unsure about ending the relationship), 2) contemplation (the woman wants to end the relationship, but has not done any thing to prepare to end it), 3) preparation (the woman wants to end the relationship and has begun preparing to do so), 4) action (the relationship ended less than six months ago), and 5) maintenance (the relationship ended more than six months ago; Figure 1). Consistent with the transtheoretical model, a variety of counseling activities were created for each stage to activate the influencing factors (e.g., self-knowledge, decisional balance) thought to move a client through the stages. A manualized WISH curriculum was built around these

Figure 1.

Staging algorithm (adapted from Burke et al., 2009).

concepts, incorporating existing published domestic violence literature and tools (e.g., the Domestic Abuse Intervention Project's Power and Control Wheel¹⁶). We also obtained input on the program from several domestic violence experts and counselors at Johns Hopkins during the development process.

At each session, a counselor first assesses a woman's risk using the widely used Danger Assessment tool¹⁷ that contains 15 items about the level of risk, which have been demonstrated to predict a woman's risk of being killed by her partner. If a woman's answers reflect a high level of risk, the counselor provides immediate crisis intervention (e.g., shelter place, police involvement) based on the individual woman's preferences. For women who are not in immediate danger, the counselor assesses a client's stage of change using the stage definitions, and the client identifies her personal goals for herself and her relationship. The counselor then identifies the appropriate influencing factors, consistent with the woman's stage and personal goals, and selects a counseling activity associated with the identified influencing factor. The WISH curriculum contains 16 counseling activities that counselors can use with their clients (Table 1 provides selected examples of the linkages between goals, objectives, stage, influencing factor, and counseling options). Safety planning and linkages to needed community services and programs (e.g., housing assistance, GED courses) are also included as part of WISH, and used at the counselor's discretion.

The Partnership

Heartly House, Inc. (HH), is a Frederick, Maryland, community-based organization that primarily serves women experiencing domestic violence or sexual assault. Its services include a 24-hour crisis hotline, emergency shelter, individual and group counseling, advocacy and legal representation, transitional housing, community outreach, professional education, and abuse intervention groups for perpetrators. HH staff expressed interest in using the WISH counseling program in response to hearing a presentation by the Johns Hopkins Center for Injury Research and Policy (JHCIRP) researchers (A.G., S.I.). Although the curriculum could not yet be disseminated as a best practice because of a lack of efficacy testing through a large-scale intervention trial, the HH counselors felt that the manualized counseling curriculum would be helpful to their efforts. The counseling HH provided before WISH was similar in that it was generally informed by the transtheoretical model. The WISH protocol, however, provided counselors with a new, structured approach to their counseling. Therefore, JHCIRP and HH agreed that they would collaborate to implement and evaluate WISH for feasibility and potential impact on clients in this setting. Both organizations felt that the results could help to enhance counseling services, and compile preliminary data for a large-scale intervention trial in the future.

The Process

Starting in September 2013, JHCIRP and HH worked together through several phases (planning, data collection, and data analyses) to complete the implementation and evaluation.

Planning involved multiple phone and in-person meet ings to create a memorandum of understanding between the two organizations, make decisions about implementation and evaluation protocols and procedures, and obtain institutional review board approval from Johns Hopkins Bloomberg School of Public Health and approval from the HH Chief Operating Officer and Board of Trustees. During this time, two HH masters-level, board-licensed clinical social workers (L.M., S.C.S) familiarized themselves with the WISH program and worked with the JHCIRP team to tailor the counseling activity scenarios as needed to be more relevant to their clients. Because the

Table 1.

Selected Example of Stage-based Counseling Activity from WISH Manual

original curriculum was created for use with clients recruited from urban HIV/AIDS treatment clinics and drug treatment programs for women, some of the scenarios used in certain counseling activities were not well-suited for the women coming to HH who were living in a more suburban setting and were otherwise generally healthy. Therefore, the team edited some of the content to be better tailored to their clients. Although there was no formal process to standardize and assess how each of the counselors was delivering the intervention, there were several in-depth conversations with JHCIRP before implementation to discuss how to use different counseling activities with clients. Once this process was completed, the two counselors began using the curriculum with all of their clients, including those women who agreed to be in the evaluation.

Together, JHCIRP and HH agreed on the evaluation design and protocols. A single group pretest/post-test design with two follow-up assessments was chosen because of limited resources and the central interest in establishing the feasibility of implementing a curriculum-based counseling program in this setting in preparation for a larger intervention trial. Direct service staff from another department at HH (M.P., M.T.) managed all of the recruitment, enrollment, consent, and data collection so that the counselors remained blind to which of their clients were enrolled in the evaluation. JHCIRP staff (J.C.) conducted in-depth key informant interviews with the HH counselors at the end of data collection.

During the data collection period (January 2014 to December 2014) JHCIRP and HH held monthly phone meetings about intervention delivery and data collection progress. HH also periodically provided real-time feedback, asked questions about specific counseling activities, and clarified any data collection requirements via email.

Near the end of data collection, JHCIRP presented HH with preliminary results, and collaboratively made decisions about final follow-up efforts and how to best present the data to be most useful to HH. JHCIRP conducted qualitative and quantitative analyses of data with input and review provided by HH.

The Evaluation

Thirty women, 18 years of age or older from the Frederick/Greater Baltimore, Maryland, area who had experienced IPV and presented to HH for IPV-related counseling were recruited to participate in the evaluation. Participants completed 1) a baseline survey (at the beginning of counseling), 2) an exit survey (after completing eight counseling sessions), and 3) a post-counseling survey (8 weeks after the exit survey). Participants provided written consent and received a $10 gift card after completing the baseline and post-counseling surveys.

Implementation Evaluation. To evaluate the feasibility of delivering WISH, we assessed program exposure/fidelity using counseling session encounter forms completed by the counselors, counselor experience using the in-depth key informant interviews, and participant experience using exit survey data. These sources provided a mixture of quantitative and qualitative data, which were analyzed using descriptive statistics and thematic coding.

Outcome Evaluation. To assess the potential impact of WISH, the present study measured six outcomes with previously used instruments: 1) overall health rating,¹⁸ 2) quality of life rating,¹⁸ 3) self-efficacy¹⁵ for staying safe from and ending/staying out of the abusive relationship, 4) frequency of IPV incidents¹⁹, including physical violence, threats, insults, screams/cursing, and forced sex, 5) decisional balance¹⁵ assessing women's perceived benefits and drawbacks of ending/staying out of the abusive relationship; and 6) stage of change for ending/staying out of an abusive relationship.¹⁵ These were measured using self-administered baseline, exit, and post-counseling surveys.

RESULTS

Sample

A total of 19 women completed baseline and exit surveys and are included in the present results. Only 13 women completed the post-counseling survey and not all yielded usable data, so these were excluded. Of these 19 women, the majority were above the age of 30 (median age, 37 years), White (90%), employed at least part time (68%), and educated at least through high school (47% completed high school/GED and 37% completed post-high school education). Just fewer than one-half (47%) were married and 53% had a household income of less than $1000 per month. About one-third of women (32%) reported living with the abusive partner and were experiencing a range of types of violence at the time of counseling (Table 2). Although slightly older and more likely to be married, the 11 women who were not included in the analysis owing to drop out or not completing surveys were

Progress in Community Health Partnerships: Research, Education, and Action demographically similar to the 19 included in the analysis.

Implementation

Exposure. Twenty-two women received the planned eight counseling sessions, and eight women dropped out before completing all eight sessions; the number of completed sessions among these women ranged from one to six, with an average of three sessions. Sessions lasted an average of 62 minutes each. Based on the client encounter forms, women engaged in a variety of activities during these sessions. The activities used most often were identifying red flags for violence in scenarios (77% of women exposed), considering different scenarios to ending the unhealthy relationship (45%), assessing knowledge of violence (36%), discussing components of healthy and unhealthy relationships (32%), identifying the pros and cons of staying versus ending a relationship (27%), and working on changing negative self-statements into positive ones (27%).

The counselors noted on the encounter forms when they used other counseling modalities, including validation and reflective listening to build rapport, cognitive reframing, trauma education, and coping strategies. They also noted that they addressed specific issues that women raised, including custody issues, boundaries, grief and loss, job stress, living independently, and issues related to children.

Counselor Experience. The counselors noted in their interviews that having a structured approach to identify their client's goals and engage them in associated activities that were the most relevant and appropriate to their stage was very helpful. For example, one counselor said,

The goal-centered a pproach was very useful for me, because as a clinician, it gave me a way to think about working with clients and it gave me tools to use while I was working on specific

[issues].

The counselors indicated that the structure provided by WISH made their sessions more focused and that the process of staging clients was helpful to guide their clinical response. They also appreciated having the encounter forms to serve as helpful reminders to check on their clients' safety, past history, and to conduct the danger assessment at each visit.

However, although they found the encounter forms and goal-based activities to be useful, they also found it difficult to get comfortable with the increased number of forms and paperwork to mange in each session. One counselor said,

Sometimes in the beginning, I was fumbling through papers. So I would say getting used to the—there are a lot of things you have to sort through. We put them in binders. Sometimes it felt a little cumbersome. I knew what binder everything was in, but based on the stage and where we needed to go, sometimes it felt a little cumbersome with the client sitting there.

Table 2.

Baseline Violence Among Women Who Completed Baseline to Exit

The counselors also noted that given the diversity of their clients' experiences, not all of the examples used in WISH activities were applicable to every client. Similarly, counselors discussed how an exercise that asked clients to discuss unhealthy relationships of others that they know often resulted in clients' inability to connect those experiences with their own, leaving the counselor feeling like that particular exercise was not effective.

The counselors suggested including components that deal with grief and loss of a relationship, and to add a focus on psychological and emotional violence. They also noted that although eight sessions were enough for many clients, some needed additional sessions, and many seemed to pre fer stretching the eight sessions over 3 months instead of 2 months. One counselor noted,

I think it almost at times felt like it was more the length of time connected, not necessarily the number of sessions because thinking about the clients who came every single week, it was a shorter relationship and … not that they didn't benefit from it, but there just wasn't as much that happened during those 2 months.

Client Experience. Women's ratings of their counseling experience upon completing e ight s essions w ere overall very positive. The vast majority of women (95% of 19 who completed) rated the program as "extremely" helpful and reported that it matched their expectations "extremely" or "quite a bit" (5-point Likert scale). The two most often used counseling activities—identifying red flags and considering different options for ending an unhealthy relationship—were rated as "extremely" or "quite a bit" helpful by 100% of the women who used them with their counselors.

Participants were also asked to write down their opinions about the program in open-ended responses, from which five themes emerged (Table 3). Women frequently mentioned how important it was to have someone to talk to who they could trust, was nonjudgmental, and provided support. Benefits identified from counseling included increased self-awareness that improved their sense of empowerment and relationship awareness that helped them stay safe. Several women expressed the desire to have had more time with the counselor and the majority of women thought adding a support group would be helpful. A few women identified specific barriers that were not addressed in the program (e.g., inability to fix financial problems). When asked what they would tell other women about the program, the response was uniformly positive. One woman wrote, "I hope they come. It is an excellent place. They will help you heal and make you feel safe." Another wrote: "Do it. You're not alone. It will get better. You're worth it."

Outcomes

Given the pilot nature of the present data and small sample

Table 3.

Participant Experience Themes

size, the six outcome measures were not analyzed statistically and results presented are descriptive (Table 4). Across all six outcomes, there were some improvements (as defined by any improvement in score). The outcomes for which the largest percentage of women improved were perceived self-efficacy of ending or staying out of an abusive relationship (9 of 15), decisional balance (12 of 17 perceived fewer drawbacks and 9 of 17 perceived more benefits of ending/staying out of an abusive relationship), and stage of change (9 of 19). No women worsened in self-efficacy to end/stay out of an abusive relationship or stage of change. Five women reported fewer benefits of ending/staying out of the relationship. Four women reported more verbal threats, although three to six women reported fewer incidents across the different types of IPV. For almost all outcomes, a large percentage of women showed no change. However, in most of these cases, baseline responses showed that women began with good health ratings and at later stages of change (e.g., 10 of 12 rated overall health as "good," "very good," or "excellent"; 9 of 10 were already in either the action or maintenance stage of change).

DISCUSSION

This project provided a unique opportunity to expand the development of a theory-based counseling program for IPV survivors by testing its real-world usefulnes in partnership with the primary intended audiences for the program—professional counselors and women reached in community settings. There are similarities and differences between the present evaluation and the six studies reviewed by Sullivan et al.⁸ For instance, several of the other evaluated programs used eight counsel ing sessions, encountered retention challenges, and had small sample sizes. Several of the programs used cognitive–behavioral therapy, and some were focused on specific subgroups of women, such as those experiencing post-traumatic stress disorder or those with both sexual assault and physical abuse, making comparisons with our work difficult. WISH is unique in integrating stages of change theory-based formative research

Table 4.

Change in Outcome Measures From Baseline to Exit

a Ten of the 12 rated health as "good," "very good," or "excellent" at baseline.

b Three of the 12 rated QOL as "pretty good" at baseline.

c Five of the seven indicated "somewhat sure I could" or "very sure I could" at baseline.

d Six of the six indicated "somewhat sure I could" or "very sure I could" at baseline.

e Six of the seven indicated "rarely" or "never" at baseline.

f One of the seven indicated "never" at baseline.

g Three of the six indicated "never" at baseline.

h Two of the five indicated "rarely or "never" at baseline.

i Nine of the 10 indicated "never" at baseline.

j Each Likert response was assigned a numerical value from 1 to 4. Scores for each decisional balance category (e.g., benefits, drawbacks) were generated by adding these numerical values together. If a woman answered at least nine out of ten questions for a decisional balance category, then a score was generated. In cases where one question was missing, the average of the other nine items was used to generate a score for the missing item.

kNone of the 10 indicated were in action or maintenance at baseline.

with domestic violence survivors^14,15,18 and input from practicing counselors in its creation, and in its ability to be used flexibly and tailored to different contexts and populations.

Overall, both counselors and clients perceived the WISH intervention positively. Moreover, although the present study had a small sample size and no control group, pilot outcome data suggest that WISH has the potential to improve per ceived health, quality of life, self-efficacy, frequency of IPV incidents, and motivation to change. In particular, the present data suggest that WISH might be most successful at laying the cognitive groundwork for behavior change, as evidenced by large proportions of women improving in perceived self-efficacy and awareness of pros and cons of relationship change. Perhaps unsurprisingly, these cognitive elements might be easier to change relative to the behaviors of an abusive partner, as assessed by IPV incidents.

The present study highlights several benefits of academic–community collaborative research partnerships. First, the partnership between HH and JHCIRP expanded organiza tional resources. A lack of resources can inhibit successful implementation of evidence-based interventions. The present partnership capitalized on each partner's preexisting skills and infrastructure to enable the project to be conducted with virtually no new financial resources. In terms of human capital, HH provided experienced counselors and additional direct service staff who facilitated intervention delivery and assistance with data collection. Similarly, JHCIRP contributed design, analysis, and troubleshooting expertise. This collaboration drew on the unique strengths of each organization and functioned on equal input and decision making, which allowed for an efficient implementation and evaluation of WISH.

Second, the partnership emphasized a "practice-based evidence" design facilitating implementation. Many projects may show promising results under very controlled circumstances, but fail to do so in more complex real-world settings. Green and Glasgow²⁰ argue that the scientific community is often overly concerned with "evidenced-based practice" rather than "practice-based evidence," which addresses external validity to support claims of implementability and generalizability between settings. One of the aims of this project was to evaluate whether the WISH intervention could feasibly be implemented and influence outcomes for clients in a community domestic violence organization, a setting with a high need for tools to help its clients. Thus, one of the project's strengths, which originated from its collaborative nature, was its emphasis on real-world usefulness.

Finally, the evaluation was fueled by strong community partner investment. The evaluation was initiated by the community partner approaching the academic partner after a presentation on WISH, and expressing interest in integrating the intervention into its current practices. This is significant because it belies the stereotype (and perhaps too often, the reality) that academic institutions implement and evaluate interventions in a top-down fashion. In this case, HH identified its own needs and reached out to fulfill those needs by requesting collaboration. At the time of publication, HH was still using components of the WISH protocol, despite the end of the pilot evaluation. HH staff played an active role at all stages of the evaluation, consistent with aspects of a community-based participatory research approach, wherein there is an "iterative process that supports co-learning and empowerment."^9,21,22

The limitations of the work center on the fact that, despite the significant human capital devoted to the project, resources of time and money were very limited and required us to use a less rigorous study design than desired. The work, however, was strengthened by a detailed focus on intervention implementation and the addition of a qualitative component that solicited feedback from both staff and participants. Future work can build on the findings of this project by refining the curriculum to include elements that were suggested during this implementation, expanding the sample size, adding additional recruitment sites and a comparison group, and increasing retention in longitudinal follow-up data collection.

CONCLUSIONS

The present study highlights the strengths of academic–community partnerships to conduct meaningful research that can advance theory-grounded, practice-based evidence. Specifically, these partnerships facilitate implementation by expanding resources and increasing engagement, as well as creating opportunities to test ecological validity. A manualized treatment protocol coupled with experienced clinicians has the potential to positively impact a variety of IPV-related health outcomes. Further scaled-up collaborative research should be conducted to rigorously examine the efficacy and effectiveness of these protocols.