Evaluation of Policy Simulation Models by Robert E. Pugh (review)

BOOK REVIEWS Evaluation of Policy Simulation Models. By Robert E. Pugh. Washington, D.C.: Information Resources Press, 1977. P. xv+320. $30.50. The use of mathematical models for policymaking in biomedical research programs and medical administration has had its ups and downs over die past decade. On the face of it, mathematical models (particularly in a computerized form) would seem to offer a viable alternative to die usual "brownie-point" decision making that goes on in most professional areas. It is attractive, in principle , to substitute a coldly objective mechanism for reaching an optimal decision for the inherently political process of finding a compromise that wiU satisfy a committee of professionals by exchange of brownie points. A mathematical model can be constructed to reflect the benefit-to-cost balance for the patients involved or for the public (rather than the benefit-to-cost situation for the decision makers). However, in actual medical and scientific decision making the use of mathematical models is relatively rare at present. The uses tend to be limited to unimportant or hypothetical planning situations. Is the reluctance to use mathematical models in policymaking decisions in the medical and scientific areas due to the inadequacies of the models, of die decision makers, or of the medical and scientific communities? Is it simply that the idea is too new to have won acceptance in the inherently conservative professional areas? Is it simply lack of availability of useable models when decisions are to be made? Is it due to overselling or underselling of this approach to policymaking? The idea is not really new. In my 1953 book, Designfor Decision [1], the concepts of statistical decision making developed in the 1940s were presented in a simplified procedures for drawing conclusions from data. Some of these procedures—controlled clinical trials, for example—have become generally accepted and are widely used in the medical area. Policymaking procedures have not. However, in 1953 clinical trials were almost as new and unfamiliar as the policymaking procedures. Since the conceptual bases of drawing medical conclusions and of making medical policies are not very different, one might wonder : Why did one set of ideas "catch on"? Why did another and similar set fail to make much progress in a quarter of a century? In one sense die book under review helps to provide an answer to this question . However, it is the book itself, radier than what is said in the book, that is informative. What is being considered in this work is a special kind ofmodel that is common in operations research, systems analysis, and other areas with a focus on policymaking—a "simulation model." The particular model that is being inPermission to reprint a book review printed in this section may be obtained only from the author. Perspectives in Biology and Medicine ¦ Summer 1978 \ 629 vestigated is an "urban dynamics" model. It is a more modest cousin of the "Umits ofgrowth" model which made headlines because ofdie end-of-the-world forecasts it produced. The author tries hard to make an even-handed neutral evaluation of the urban dynamics model. This effort to avoid prejudgment of the model means that the author takes 179 pages of text (in die 192 pp. of the main text) to get to the point: "Although the evaluation reasonably concludes that urban dynamics is inadequate for policy analysis use, the evaluation is far less satisfying in pinpointing a concise reason why the model is unsatisfactory. Basically, this is because the model is inadequate for a very broad reason: It floats free of the real world." This statement brings into sharp focus the basic difference between a genuine scientific model, mathematical or otherwise, and the kinds of models that are commonly devised by pure mathematicians, by computer-oriented systems analysts, or by economists. The prime directive of modern science, the Galilean rule, applies to any dieory: A theory must fit the facts. It makes no difference whedier a theory is expressed in an algebraic language or in some verbal technicaljargon , it must be firmly anchored to reality. In genuine science, thefirst step (not the last step) in the evaluation of a mathematical model (which is simply a theoretic construct...