Measuring Fidelity and Interrater Reliability of Illness Management and Recovery Groups in State Psychiatric Hospitals

Fidelity scales can be used to evaluate adherence to specific standards for manualized group interventions. This study examined interrater reliability among nine independent raters of the Illness Management and Recovery (IMR) Fidelity Scale when IMR groups were implemented in five state psychiatric hospitals. Fidelity reviews were conducted on randomly selected groups and interrater reliability was assessed. IMR fidelity scores ranged from 2.7 to 3.5. The IMR fidelity scale was found to have adequate interrater reliability with 16 pairs of independent raters (intraclass correlation coefficient of 0.84). The findings expand knowledge about the psychometric properties of the IMR Fidelity Scale. Implications for group counseling are discussed.