Does Measurement Measure Up?: How Numbers Reveal and Conceal the Truth

Preface

PREFACE The British comedian Eddie Izzard launches his stand-up comedy routine in Glorious (1997) by apologizing to the audience for the presence of a video crew, with their cameras and lights. To make up for this, Izzard promises, ‘‘I’m going to be extra-funny tonight—an extra 10% funny.’’ Then he adds, mockingly, ‘‘You can’t check, can you?’’ Of course not. It is absurd to think that one could really measure how funny something is. Or is it? Hollywood television and movie producers have developed sophisticated ways to measure how funny something is. Millions of dollars, in fact, ride on their ability to measure ‘‘funniness.’’ But lots of other things depend on measurement. Will your son be admitted to his college of choice? That depends on his high school grade point average and SAT scores—measures of his intellectual ability and academic accomplishments. Will our government regulate carbon dioxide emissions, perhaps drastically changing our transportation systems and energy infrastructure? That depends on a bewildering jumble of complicated measurements that form the core of the global warming debate—things like atmospheric and oceanic temperatures and CO≤ levels—and on how those measurements are analyzed and interpreted by the experts. Will Alan Greenspan and his colleagues at the Federal Reserve Board raise or lower interest rates at their next meeting? That depends on how they interpret the trends in various measurements of the national and global economies—things like the gross domestic product, the unemployment rate, and the balance of trade. Will your alma mater’s football team, an undefeated powerhouse, be P REFACE x invited to play for the national championship? That depends on your team’s Bowl Championship Series (BCS) rating—a measurement of ability based on wins, losses, and the strength of a team’s opponents. Will a man convicted of rape and murder be released from prison after ten years on death row? That depends on whether measurements of various markers in his DNA match those of bodily fluids left at the crime scene. Will your doctor prescribe powerful, expensive drugs to help clear your arteries of dangerous deposits? That depends on measurements of both the ‘‘good’’ and ‘‘bad’’ levels of cholesterol in your blood and how those measurements change over time. The British physicist William Thomson, better known as Lord Kelvin (1824–1907), claimed that without measurement, our knowledge of something —anything—is ‘‘meager and unsatisfactory.’’ I have often thought about Kelvin’s words over the years. As an engineering student, I began to master my chosen profession through quantification—by measuring things. I learned to think, not ‘‘The bridge is really strong’’ (meager and unsatisfactory knowledge), but rather ‘‘The bridge has a safe load limit of 10 tons’’ (now I really know something about the bridge). That is a simple enough example. But how should I think about the abilities of a colleague, who may be the engineer who designed that bridge? Should I consider that she is highly intelligent, creative, diligent, and tenacious? Would my opinion be altered if I knew that in college she only had a 3.2 grade point average, and that she made a 1250 on the SAT? Is the former meager and unsatisfactory with respect to the latter? Or vice versa? This book is an exploration of the ways in which knowledge and measurement are interrelated. It is a book about what we measure, why we measure, and how we measure. Measurement, it would seem, is much broader and more pervasive than Lord Kelvin imagined. Kelvin believed that, in order to know something about anything, you have to be able to measure it. The Greek mathematician Philolaus put it even more bluntly in the fifth century BC: ‘‘Everything that can be known has a number.’’ Twenty-five centuries later, it seems as though we live in a world ruled by numbers, from IQ to pain-level measurements to ‘‘impact factors’’ that rate the importance of a particular piece of research. Sorting out which measurements have meaning and which do not is a skill. The sheep in George P REFACE xi Orwell’s Animal Farm end up bleating, ‘‘Four legs good, two legs better.’’ That is a measurement too—with a value judgment placed on it. In the end, this book asks us to stop and take a look at the world around us from a slightly different perspective—the perspective of measurement. Lord Kelvin...