The PerformanceStat Potential: A Leadership Strategy for Producing Results

10. Conducting the Meetings

9 Analyzing and Learning from the Data How can the data be analyzed to diagnose performance deficits, to nominate opportunities for improvement, to identify what approaches (if any) are working and why, and to suggest new strategies? “God has chosen to give the easy problems to the physicists.” charles lave, University of California, and james march, Stanford University1 “Discovery consists of seeing what everybody has seen and thinking what nobody has thought.” albert szent-györgyi, 1937 Nobel Prize winner in medicine2 Robert Dunford, superintendent-in-chief of the Boston Police Department, opened the department’s biweekly CompStat session with a question: “What’s going on on the street?” Shootings in Boston were off, and Dunford wanted to learn why. As he told the department’s 50 top managers, “If we can identify what we are doing, we can replicate it.” Dunford’s question provoked a variety of answers: “Maybe the drug units” were having an impact, noted one officer. “There’s hardly anyone at the usual spots,” observed another. One district commander suggested that it might be “aggressive patrol.” “Quicker indictments” proposed another, explaining that the grand jury was working better so that someone arrested on Friday night would be indicted on Tuesday, not six months later. Indeed, Dunford’s question—and his search for learning—generated numerous explanations that could be organized into three broad categories: 1. It was the weather. During the previous weeks, Boston had experienced a lot of rain. 145 09-2527-5 ch9.indd 145 4/10/14 4:28 PM 146 Analyzing and Learning from the Data 2. It was the police—particularly aggressive patrol and quicker indictments that were getting the “high-impact players” off the street. 3. It was purely random. Noted one officer: “Crime goes up. Crime goes down.” As he had done before, Dunford was trying to get his department’s leadership team to think analytically—to learn from the available data. Not everyone, however , was prepared to do so. The Data Never Speak for Themselves These multiple explanations for this drop in shootings debunk the seductive cliché: “The data speak for themselves.” For data can “speak” only through a framework, a theory, a cause-and-effect concept, a perception of the world and how it works. All frameworks provide a way of thinking, and the world is full of them. When we analyze problems, we use a framework—usually implicitly and unconsciously—to organize and structure our thinking. Long division is such a framework that is useful in a variety of circumstances. For this framework can, by creating a comparison, help us decide whether a number is big or small. Economics provides another framework. So does psychology. Both help us think about how people behave. Yet our conclusions about any specific observation or prediction of human behavior may depend on which framework we choose. Analytical frameworks can be powerful. “Practical men, who believe themselves to be quite exempt from any intellectual influences,” wrote John Maynard Keynes, “are usually the slaves of some defunct economist.”3 “The thinker,” observed Oliver Wendell Holmes, Jr., knows that “after he is dead and forgotten, men who never heard of him will be moving to the measure of his thought.”4 People may believe that they are analyzing their data from their own unique perspective . But they would be wrong. Data are collections of abstract digits: 7, 9, 4, 6 . . . (or, inside a computer, ones and zeros). To interpret the data, we need a framework—a lens through which to observe the data and extract from the otherwise incomprehensible gibberish some knowledge: a coherent pattern, a valuable lesson, a revealing causeand -effect relationship, an implication for action. For decades, astute observers have railed against this data-speak-for-themselves nonsense. In 1932, Carl Becker, then president of the American Historical Association , observed that “to suppose that the facts, once established in all their fullness, will ‘speak for themselves’ is an illusion.”5 “Be aware of the intellectual traditions and choices out of which the ‘data’ emerge,” cautioned Gary Marx, 09-2527-5 ch9.indd 146 4/10/14 4:28 PM Analyzing and Learning from the Data 147 professor emeritus at MIT. “The facts do not speak for themselves. Look for the ventriloquists in the wings.”6 When James Heckman accepted the Nobel Prize in economics, he was unequivocal: “The data do not speak for themselves.”7 Yet, the cliché lives...