Financing Low Income Communities

9. Measuring the Impact of Community Development Financial Institutions' Activities

CHAPTER NINE MeasuringtheImpactofCommunity DevelopmentFinancialInstitutions’Activities RobinsonHollister THIS CHAPTER reviews both attempts to evaluate the impacts of community development financial institutions (CDFIs) and methods of evaluation. Some may find it highly negative about whether the impact can be measured. I therefore begin by explaining the rationale for presenting the material this way.1 It seems that many of those calling for estimates of the impact of CDFIs— primarily funders—have unrealistic expectations about both the magnitude of change CDFIs are likely to be able to achieve with their investment activities and the degree to which reliable estimates can be obtained of those changes that might have occurred. To expect a CDFI to have a measurable, sizable effect on the degree of poverty in a given area is unrealistic. As John Caskey has pointed out, the assets of a CDFI are at most about the size of those of a small community bank, and we would not expect such a bank to have a substantial effect on community poverty (conversation with author).2 As in many fields, at an early stage, those promoting a concept or program make extravagant claims about the likely or actual effects such activities will generate. This has been the case with CDFIs. A first step in trying to help formulate more realistic expectations about “provable” impacts of CDFIs is to show how these misleading, casual, and extravagant claims are often generated. Second, the distinction between an outcome and an impact is often not well understood. It seems important that those asking for and getting evaluations of CDFI activities be aware of this distinction so they can properly assess what it is they are getting from the evaluation product. 266 FinancingLow-IncomeCommunities Third, few realize how very difficult it is to establish causality rigorously. One may be provided what is called an estimate of the impact of a CDFI activity , but to determine rigorously that the CDFI actually caused that configuration of events is a very demanding task that will be achievable only in a very limited set of circumstances. Given this set of rather discouraging considerations, it is important to recognize that there are still worthwhile things that can be learned from quantitativeanalysisrelatingoutcomestoparticularCDFIactivities .Therefore,this chapter suggests when and with what methods evaluation and monitoring might partially fill the void. We hope this will help those posing impact goals for CDFI activities they are funding or programs they are managing to formulate more realistic expectations about what it is that an evaluation could provide them. Further, this discussion will help consumers of CDFI evaluations better assess whether the estimates from an evaluation in fact indicate howclosetothegoalstheCDFIactivityhascome.Theseconsumersshouldnot be asking programs to prove this when there is not an available method that meets a high proof standard. At the same time, they should not be persuaded by flawed estimates that are offered as proof. ALITTLEHISTORY:PRIS,CDFIS,ANDEVALUATION One component of the War on Poverty efforts in the 1960s was community action, efforts that focused on the community rather than the individual and were often referred to as place-based strategies. Although some elements of that agenda survived the general discouragement over how effective such efforts were, new place-based efforts in the 1970s took the form of community development corporations (CDCs). Some of the most noticed of these—Bedford-Stuyvesant being one example—mixed attempts to attract industry, and thereby employment, to the communities with infrastructure improvements, most notably housing.3 A chronologically parallel development was a provision of the Tax Reform Actof1969,whichallowedfoundationstomakeinvestmentsto“supportcharitable activities that involve the potential return of capital within an established time frame” (The Foundation Center 2005). These are called program related investments (PRI). Some foundations began to mix PRIs with the normal grant programs. Often, however, those making PRI investments were in administrative or finance offices and had limited interaction with grant program officers . These developments brought the perspective of financial analysis and management into the social programming area. Eventually, after some notable misadventures with direct investments in particular enterprises, foundations began to realize that specialized skills were needed to assess potential PRIs; eventhoughahigherriskprofilewastobeexpectedinthePRIportfolio,sound assessments of the likelihood that capital would be returned called for different CommunityDevelopmentFinancialInstitutions’Activities 267 types of analysis. On the one side, finance officers were not adept at assessing program content. On the other, grant officers couldn’t handle the financials. In the later 1970s and early 1980s, foundations began to turn to intermediary...