Evaluating the Validity of Systematic Reviews to Identify Empirically Supported Treatments

The best available evidence is one of the three basic inputs into evidence-based practice. This paper sets out a framework for evaluating the quality of systematic reviews that are intended to identify empirically supported interventions as a way of summarizing the best available evidence. The premise of this paper is that the process of reviewing research literature and deriving practical recommendations is an assessment process similar to the assessment process that we use to understand student performance and derive educational recommendations. Systematic reviews assess the quality and quantity of evidence related to a particular intervention and apply standards to determine whether the evidence is sufficient to justify an endorsement of the intervention as "empirically supported". The concepts and methodological tools of measurement validity can be applied to the systematic review process to clarify their strengths and weaknesses. This paper describes ways in which these concepts and tools can be brought to bear on systematic reviews, and explores some of the implications of doing so.