pdf Download PDF

Public Reasoning and Health-Care Priority Setting:
The Case of NICE
ABSTRACT

Health systems that aim to secure universal patient access through a scheme of prepayments—whether through taxes, social insurance, or a combination of the two—need to make decisions on the scope of coverage that they guarantee: such tasks often falling to a priority-setting agency. This article analyzes the decision-making processes at one such agency in particular—the UK’s National Institute for Health and Care Excellence (NICE)—and appraises their ethical justifiability. In particular, we consider the extent to which NICE’s model can be justified on the basis of Rawls’s conception of “reasonableness.” This test shares certain features with the well-known Accountability for Reasonableness (AfR) model but also offers an alternative to it, being concerned with how far the values used by priority-setting agencies such as NICE meet substantive conditions of reasonableness irrespective of their procedural virtues. We find that while there are areas in which NICE’s processes may be improved, NICE’s overall approach to evaluating health technologies and setting priorities for health-care coverage is a reasonable one, making it an exemplar for other health-care systems facing similar coverage dilemmas. In so doing we offer both a framework for analysing the ethical justifiability of NICE’s processes and one that might be used to evaluate others.

1. INTRODUCTION

Health systems that provide for universal patient access through a scheme of prepayments—whether through taxes, social insurance, or a combination of the two—need to make decisions on the scope of coverage that they secure. Such decisions are inherently controversial, implying, as they do, that some patients will receive less than comprehensive health care, or less than complete protection from the financial consequences of ill-heath, even when there is a clinically effective therapy to which they might have access. [End Page 107]

Controversial decisions of this sort call for a public justification for covering or not covering a given treatment. Priority-setting agencies play a key role in providing such a justification. A recent WHO report recommended that national health-care systems set up institutions to make reasonable and accountable decisions on the path to universal health coverage (WHO 2014). In a growing number of countries, governments have already established expert bodies to this effect (Glassman and Chalkidou 2012). Moreover, even in countries that explicitly reject considering cost effectiveness in coverage decisions, this stance is now being questioned. For example, one of the most influential medical societies in the U.S., the American Heart Association, recently announced that they would begin to use cost data to rate the value of treatments in their clinical practice guidelines and performance standards (Cardiosource Website 2014; Anderson et al. 2014), arguably because system-wide solutions for determining the scope of coverage are lacking (Pollack 2014).

In this paper, we analyze the process of public reasoning of one priority-setting agency in particular, the UK’s National Institute for Health and Care Excellence (NICE). We offer this case study for a number of reasons. First, since its establishment in 1999, NICE has secured a high reputation for its work among informed observers in the UK and abroad (Culyer 2006; Hill et al. 2003), attracting “international policy tourists” (Marmor and Klein 2012, 330). Although sometimes criticized, its processes of decision making are widely respected, not least for their rigorous approach to the assessment of value for money in health care. Indeed, NICE International, the institute’s global arm, plays an active role in advising other countries interested in setting up bodies for making explicit coverage decisions (NICE Website 2014a).

Second, NICE has also been praised for its recognition of various procedural and substantive ethical values in its decision-making processes. For example, in respect to the former, Gutmann and Thompson (2004) cite NICE as operating an exemplary decision-making process in their influential discussion of the idea of deliberative democracy. In respect to the latter, NICE has also been praised for recognizing a wide range of what it terms “social values”—like fairness, nondiscrimination and responsiveness to need—as necessary components of legitimate decision making (Littlejohns and Rawlins 2009; NICE 2005; NICE 2008).

Third, in its sixteen years of existence, NICE has completed a large volume of work and, correspondingly, been the focus of a large amount of external commentary. Between 2000 and February 2014, NICE published [End Page 108] 306 technology appraisals ranging across interventions for conditions as diverse as cancer, multiple sclerosis, depression, and anxiety. It has thus built up a considerable body of “case law” alongside the formal statements of its decision protocols, much of which has prompted political mobilization by patient groups, vocal opposition by industry, hostile coverage in the press, and controversy among parliamentarians, political activists, and academics—especially when NICE’s decision has been negative and coverage for an intervention by the National Health Service (NHS) has not been recommended. NICE’s body of work and the debate it has engendered therefore offers a rich resource for commentary and analysis. More importantly, as we shall show, the intellectual development and modification of its policy principles provide us with important insights into practical public reasoning and health-care priority setting.

In this article we evaluate NICE’s approach to priority setting, considering the extent to which its approach is reasonable. In this, we depart from the standard measure against which most priority-setting bodies are evaluated, namely the Accountability for Reasonableness (AfR) framework (Daniels and Sabin 2008). On AfR, the justifiability of priority-setting mechanisms is taken to consist almost exclusively of their procedural virtues; that is, the extent to which they are accountable for their reasonableness. In our analysis, by contrast, we evaluate the substantive values of NICE’s methods against the Rawlsian notion of reasonableness—namely, the willingness to propose fair terms of cooperation and abide by them provided that others do (Rawls 1996, 48–54). To be sure, the Rawlsian notion of reasonableness also underpins AfR and its “relevance” condition in particular, which requires that health providers give a reasonable explanation of how they aim to meet varied health needs under reasonable resource constraints (Daniels 2008). However, our use of the test is more basic: we consider how far the values used by a body such as NICE may be reasonable in a Rawlsian sense, regardless of whether the body itself might be accountable by Daniels and Sabin’s definition. Of course, despite this difference, there is a sense in which our analysis complements Daniels’. That is, insofar as it shows some of NICE’s reasons to be reasonable, it also identifies a certain set of reasons as relevant reasons for action within AfR—aiding Daniels’ account on a point where he has thus far been somewhat vague (Rid 2009). Altogether then, this essay offers three things: a fine-grained account of how far NICE’s approach to decision making may be considered reasonable; a new structure for assessing the ethical justifiability of public reasoning in priority-setting agencies in general and, finally, the [End Page 109] identification of a certain set of reasons as relevant reasons for action in the distribution of health-care resources.

Our argument proceeds as follows: in Section 2 we set out the nature of NICE’s process. We argue that NICE’s use of evidence on a given technology’s cost effectiveness is often wrongly described as a maximizing strategy. Instead, we suggest that NICE’s use of cost-effectiveness data is best understood as a satisficing strategy, wherein NICE assumes a presumption not to fund interventions the cost effectiveness of which (compared to an existing treatment) is below a minimum threshold. We also stress that in both theory and practice, NICE takes this presumption to be rebuttable. That is, the presumption to fund interventions below a given cost-effectiveness threshold can be overturned by considerations drawn from public social or ethical values and by the considerations implicit in NICE’s developing “case law,” the cumulative set of decisions that the institute has made over time. In Section 3, we set out the notion of “reasonableness” against which we aim to evaluate NICE, its relevancy as a standard of evaluation in this domain and why, at a basic level, it would be unreasonable for health systems not to have at least some body that sets priorities for health-care coverage based on publicly justifiable principles. In Section 4, we then show that, as well as it being generally reasonable that some body is instructed to set priorities for health-care coverage, NICE’s method of prioritizing treatments for coverage—its rebuttable presumption to fund interventions below a given threshold of cost effectiveness—is also reasonable. This argument shows that NICE’s overall approach to prioritization decisions is reasonable. However, we also note its specifics are more questionable. In the last section we question three aspects of NICE’s approach: its choice of the cost-effectiveness threshold and the rationale for that choice; its process of choosing certain social or ethical values as grounds for departing from that threshold; and the way it adjusts that threshold in light of those social and ethical values.

2. NICE’S APPROACH TO PRIORITY SETTING: THE PRESUMPTION OF COST EFFECTIVENESS

NICE is responsible for a variety of functions within the English NHS, clustered around the production of guidance.1 The core elements of NICE’s work, though, are its technology assessments—that is, its guidance to NHS purchasers on whether a given intervention (typically pharmaceuticals, devices, diagnostics, surgical and other procedures, and health promotion tools) constitutes a “good buy.” In theory, if not in [End Page 110] practice, these technology appraisals potentially have a profound impact on the range of services and treatments the NHS provides. If an intervention passes NICE’s evaluation process, then it is recommended for use within the NHS and those commissioning services have an obligation to make them accessible to the populations whom they serve (the NHS Constitution giving patients the right to services and treatments recommended by NICE [Department of Health 2013]). If the intervention is not recommended, NHS purchasers are still free to fund that intervention if they so wish but they are not obliged to do so.

In the first instance, NICE’s appraisal agenda is set by the Department of Health (DoH). However, the Institute also has a significant autonomy in deciding exactly how the issues for any one appraisal are determined and it operates with expert panels for appraisals that it constitutes and convenes. Once the initial choice of technologies has been made, NICE undertakes a “scoping” process to determine the appropriateness of the proposed remit and defines the specific questions that each technology appraisal will address. Next there is an “assessment” process, wherein an independent academic group is tasked with a systematic evaluation of the relevant evidence available on a technology (including evidence pertinent to its cost effectiveness), either from published data or the submission of a report from industry. Finally, there is an “appraisal” process, in which an Appraisal Committee considers evidence contained in the reports and analyses produced in the assessment phase and additional information supplied by consultees, commentators, clinical specialists, patient experts, and commissioning experts. In addition, NICE’s “Citizen’s Council” provides a regular review of values for use in the appraisal of new technologies (see Table 1). [End Page 111]

Table 1. NICE’S APPRAISAL PROCESS
Click for larger view
View full resolution
Table 1.

NICE’S APPRAISAL PROCESS

In the context of these procedures, NICE uses a decision protocol prioritizing cost-effectiveness data. NICE’s assessment of cost effectiveness contains the following elements (NICE 2013):

  1. 1). Evaluation of effectiveness: NICE seeks to identify the benefits arising from a health-care intervention. The benefit is defined as the average improvement in the health status of individuals receiving the intervention over and above any other gain they might receive. The key measure of this benefit is the gain in Quality-Adjusted Life Years (QALYs) that a typical individual derives from an intervention. Health benefit is therefore measured by reference to both an extension of life and an improvement in the quality of life.

  2. 2). Evaluation of cost: In the case of technology appraisals, costs are total NHS and personal social service costs. For medicines and devices, costs are given by the published list price. In recent years there has been some modification of this arrangement in the case of “patient access schemes” negotiated between industry and the DoH. These are innovative pricing agreements designed to improve cost effectiveness and facilitate patient access to specific drugs or other technologies. However, such schemes are usually initiated when draft guidance from NICE suggests that the intervention is not cost-effective. [End Page 112]

  3. 3). Evaluation of cost effectiveness: NICE evaluates an intervention by reference to its “incremental cost-effectiveness ratio” (ICER). NICE defines the ICER as “the ratio of the difference in the mean costs of an intervention compared to the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes” (NICE 2008). Although NICE has explicitly denied having a maximum value of an ICER above which it does not recommend an intervention (NICE 2002), based on its decisions to date, commentators have surmised that its normal threshold cost per QALY gained (i.e. prior to any other consideration) is between £20,000 and £30,000 (Towse 2002).

It is sometimes claimed that NICE’s use of cost-effectiveness data in evaluating health-care technologies equates to a “maximizing” strategy. Indeed, NICE itself might be said to encourage such a reading. For example, in its document on Social Value Judgements, NICE states that it incorporates a maximizing approach into its work as an expression of a utilitarian approach to securing justice in the distribution of health-care resources (NICE 2008). Similarly, Alan Williams (2004, 6)—an influential figure in the establishment of NICE—argued that technology assessments were “driven by the objective of maximizing population health subject to a budget constraint” (cf. Schlander 2008; Neuman and Greenberg 2009).

However, even if NICE’s privileging of cost-effectiveness data could be understood to follow from a “nonwelfarist” or “extra welfarist” normative position (cf. Shah et al. 2012, 158), its actual use of such data does not equate to a maximizing approach. First, NICE does not maximize QALYs in the NHS because it makes no effort to identify and prioritise those interventions that offer the least cost per QALY gain out of all those interventions that might be funded, say, drawing up a league table of all relevant interventions and setting them in some order of priority, much like the first stage of the Oregon health-care plan (Tengs et al. 1996; see here also Alan Williams’ suggestion that this is really what NICE ought to do—2004, 16–18). Instead, NICE assesses technologies one by one, making piecemeal judgements about whether a particular intervention meets its threshold degree of cost effectiveness, irrespective of whether that intervention offers the least cost per QALY gain of all those interventions that might be funded. NICE’s decision rule here might be seen as analogous to Disney World’s height restrictions. To ride the Big Thunder Mountain Railroad, Disney World requires passengers must be at least 40 inches tall. However, this requirement does not mean the height of passengers will be “maximized,” given the pool of prospective riders. Instead, entrants are required to meet a specified minimum. [End Page 113]

NICE also only judges the cost effectiveness of those interventions it assesses “compared to the next best alternative (which could be no action or treatment)” (NICE 2008). In other words, a new treatment will be funded if the health gain that it provides relative to the existing standard treatment comes in at below the threshold cost. Again, this fails to equate to a maximizing strategy, for where the existing standard treatment is of low cost effectiveness, the new treatment may be incrementally cost-effective, even it would be of low cost effectiveness in absolute terms.

Here it might be argued that, while each choice NICE makes may not be optimally maximizing, the cumulative effect of its piecemeal decisions over time will lead to a defined set of interventions with the highest value—for example, as measured by aggregate health gain for a given budget. However, for this to be true, the target of its evaluations—that is, costs and benefits of the services and treatments under scrutiny—would need to remain constant over the course of the decision-making process. Needless to say, this is simply not reflective of the context in which NICE’s evaluations are made. In reality, each assessment requires substantial information gathering and processing, and during that time new technologies are developed; the benefits package is, in this sense, a moving target. In this way, then, a synoptic view of policy agendas is rendered practically impossible simply by virtue of the demands of information collection and processing—something students of public policy have recognized since the pioneering work of Simon on bounded rationality (1976; 1983), and Braybrooke and Lindblom’s work on incrementalism in policy making (1963).

Rather than viewing NICE as operating a kind of nonoptimum maximizing strategy, then, it is clearer to think of its process in terms of what Simon called “satisficing,” or “satisfying of the minimum” (cf. Lord, Laking, and Fischer 2004)—in this case, ensuring that each recommended technology fulfils a minimum cost-effectiveness threshold.

Having established data on how far a given intervention satisfies its cost-effectiveness threshold, NICE next recognizes a further set of “social values” as considerations that may justify recommending a technology whose ICER puts it over that threshold (Devlin and Parkin 2004; Rawlins and Dillon 2005; Culyer 2006; Rawlins, Barnett, and Stevens 2010). As NICE states in Principle 3 of its Social Value Judgments: “Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone. NICE must consider other factors when developing its guidance, including the need to distribute health [End Page 114] resources in the fairest way within society as a whole” (NICE 2008).2

NICE adopts a two-level process for assigning weight to “social value judgments” in its decision-making process. At one level, departure from an ICER of over £20,000 per QALY gained is considered acceptable on the basis of five considerations (described as “modifiers” in its Guide to the Methods of Technology Appraisal): i) the degree of certainty around the ICER; ii) whether there are strong reasons to doubt whether the assessment of the change in health-related quality of life has been captured adequately; iii) the innovative nature of the technology; iv) whether the technology meets the criteria for special consideration as a “life-extending treatment at the end of life”; v) aspects that relate to nonhealth objectives of the NHS (NICE 2013).

NICE takes each of these “modifiers”, then, to offer a potential justification for departing from its threshold of cost effectiveness, where Appraisal Committees need to make an increasingly stronger case for recommending the intervention (with reference to the factors listed above) as the ICER climbs over £20,000 per QALY (NICE 2013). Although NICE has never issued specific instruction to Appraisal Committees with regard to the upper limit of the additional weight they should attach to the health benefits achieved by technologies in light of such considerations, in practice the cumulative weight in circumstances where all modifiers apply has not exceeded 2.5 from a starting point of £20,000 per QALY (NICE 2014).

As well as those values specified as grounds for exceeding the ICER, NICE also uses a set of further social and ethical values outlined in its key document Social Value Judgements. Some of these follow from NICE’s institutional context. For example, under UK law, NICE has an obligation to avoid discrimination, so that factors like race or ethnicity, age, sex or gender, and sexual orientation are to be considered as relevant only if clear evidence shows them to be proxies for clinical effectiveness (NICE 2008). However, NICE also includes other values taken to be widely held in UK society, including amongst others special consideration of the needs of disabled people, special consideration of the relief of stigma, and reducing health inequalities (especially by benefiting the most disadvantaged) (NICE 2008).

While recognizing these values, however, NICE either explicitly rejects or simply ignores others (for example, individual choice or preferences, rule of rescue, treating rare conditions as special, age as a proxy for social worth, personal responsibility for health, socio-economic status) (NICE 2008). Precisely where and how those values that NICE includes [End Page 115] within its decision-making process enter into its evaluation of a given technology is not always clear. However, the influence of such values in justifying departures from a strict accounting of a given technology’s cost effectiveness is clear enough. For example, Devlin and Parkin (2004) show that cost effectiveness, uncertainty, and the burden of disease together explain NICE decisions better than cost effectiveness alone. Moreover, Table 2 gives examples where NICE has explicitly recognized certain values as influencing evaluations of particular technologies.

Overall, then, how are we best to conceptualize NICE’s process? We propose that NICE’s method is best thought of as setting a rebuttable presumption or default to prioritize the social value of cost effectiveness that can be modulated by other social or ethical values, such as reducing health inequalities (See Table 3). It is the reasonableness of NICE’s method, understood in this sense, that we seek to evaluate in the next three sections.

3. REASONABLENESS AND PRIORITY SETTING

In evaluating the very idea of a priority-setting institution like NICE, we draw on a concept of reasonableness inherited from the philosophical literature, namely Rawls’ conception of reasonableness as the willingness to propose fair terms of cooperation and abide by them (provided that others do) (1996, 48–54).

One important feature of this concept is that it allows for an allocation of resources that is reasonable and yet at the same time is based on a set of reasons about which reasonable people might disagree. A disagreement is reasonable when it is possible for reasonable and well-informed individuals to continue to disagree about the topic despite all the evidence and arguments that can be brought to bear on both sides. A key contemporary source for this view is Rawls’ (1996, 54–58) discussion of the “burdens of judgement,” according to which the complexity of empirical and normative assessment, and the way in which conflicting considerations can be balanced, mean that reasonable people will disagree.

Overall, then, the actions of a priority-setting body might be said to be reasonable to the extent that they appeal to reasons accepted as relevant by people who are trying to find ways of cooperating with each other on mutually acceptable terms, and unreasonable to the extent that they do not appeal to reasons accepted as relevant. Moreover, the actions of a priority-setting body might still be considered reasonable even if they ultimately rest on matters about which reasonable people might disagree. [End Page 116]

Table 2. THE USE OF SOCIAL VALUES TO OVERTURN PURE COST EFFECTIVENESS
Click for larger view
View full resolution
Table 2.

THE USE OF SOCIAL VALUES TO OVERTURN PURE COST EFFECTIVENESS

[End Page 118]

Table 3. NICE’S METHODOLOGY FOR EVALUATING HEALTH INTERVENTIONS # “Based on NICE’s Social Value Judgments (2nd edition, 2008) and Guide to the Methods of Technology Appraisal (2013).7,12 Other social values that have been invoked in actual NICE appraisals are not represented. * End-of-life criteria: 1) patients with life expectancy of <24 months, 2) intervention offers life extension of >3 months compared to existing alternatives, 3) intervention is licensed for small patient populations (total <7000 patients) % Age may be invoked if 1) a good indicator of patients’ health status and / or likelihood of adverse effects and / or treatment response and 2) no practical way of identifying patients other than by age. Furthermore, a lower discount rate for costs and benefits may be applied if a treatment 1) restores full or near full health in patients who would otherwise die or have a very severely impaired life and 2) the resulting health benefits are long-term (>30 years, i.e. typically applying to younger patients).
Click for larger view
View full resolution
Table 3.

NICE’S METHODOLOGY FOR EVALUATING HEALTH INTERVENTIONS

# “Based on NICE’s Social Value Judgments (2nd edition, 2008) and Guide to the Methods of Technology Appraisal (2013).7,12 Other social values that have been invoked in actual NICE appraisals are not represented.

* End-of-life criteria: 1) patients with life expectancy of <24 months, 2) intervention offers life extension of >3 months compared to existing alternatives, 3) intervention is licensed for small patient populations (total <7000 patients)

% Age may be invoked if 1) a good indicator of patients’ health status and / or likelihood of adverse effects and / or treatment response and 2) no practical way of identifying patients other than by age. Furthermore, a lower discount rate for costs and benefits may be applied if a treatment 1) restores full or near full health in patients who would otherwise die or have a very severely impaired life and 2) the resulting health benefits are long-term (>30 years, i.e. typically applying to younger patients).

[End Page 120]

Why think that it is important whether an institution such as NICE meets the criterion of reasonableness? That is, why should it matter, ethically speaking, whether NICE is reasonable or not? In our view, the moral significance of how far NICE is able to meet the reasonableness standard arises from its being an institutional element in the attempt to secure universal access to medical care.

As is the case in the overwhelming majority of modern health systems in high-income countries (and increasingly in middle-income countries), the UK secures universal access to health care by operating a system of “shared savings” (White 1995). Such a system spreads the financial risks associated with health-care costs widely among those living in the country through systems of prepayment. Two features of this model are relevant for the reasonableness of institutions such as NICE. First, in tax-funded systems, like those of the UK, spending on health care is in direct competition with spending on other politically determined goals, for example education, transport, or security. Second, the fixed budgets of these systems are insufficient to cover all conceivable and technically possible health needs. To begin with, new therapies can be expensive per unit cost. For example, Novo Seven, a recombinant therapy for haemophilia resistant to first-line treatment, costs €6,000 per dose, with doses needing to be administered every two hours until bleeding stops, a protocol that in one German case led to total costs of over 2m for one patient (Social Values Project Website 2014a). Moreover, even where unit costs are low, the widespread use of interventions that by themselves are relatively modest in cost can lead to considerable expenditure. For example, the NHS in the UK spent £321.7 million on atorvastatin in 2009 (The Health and Social Care Information Centre 2010), though its unit costs in 2005 ranged from £0.35 to £0.64 per patient per day (NICE 2006).

By virtue of the economic constraints in which it occurs, then, health-care expenditure within a system of shared savings automatically invokes the ethics of opportunity costs. Given that money spent on one set of interventions will inevitably displace resources devoted to other interventions, public authorities need to evaluate the opportunity costs involved in funding one set of health-care intervention over others. Unless a positive decision to spend on one form of treatment is set against the forgone opportunity of the same resources going to other forms of treatment, there is a risk of unfairness in the allocation. In particular, those most able to articulate their case may well be disproportionately favoured at the expense of those less articulate. [End Page 121]

For this reason, any body tasked with making decisions as to how resources are allocated is under a general duty to act reasonably. That is, at a minimum, bodies such as NICE must provide a rationale that makes clear how the social and ethical values and processes involved in such priority setting are acceptable to those seeking fair terms of cooperation. This requirement might be thought of as both a moral requirement and an obligation arising from the social contract upon which many publicly funded health-care systems are built.

Drawing on the discussion of reasonableness earlier, we can make three important points about the allocation of health-care resources. First, a scheme of shared savings in health care can be reasonable even when it entails that some interventions will not be funded. Perhaps the least contentious case of a reasonable refusal to fund a particular treatment is when funding it would displace a greater number of more effective treatments—as when a therapy is of high cost but only marginal effectiveness. Second, there can be reasonable disagreement about the social values and processes to be used in prioritizing health care. Third, we might also say that any attempt to contain costs for health care by setting explicit priorities will require a body that is similar to NICE. Such bodies may invoke different values, but in all cases there will need to be explicit appeal to publicly-held social values, to justifiable processes, and there will need to be a body to make choices. Institutions like NICE, therefore, meet a general criterion of reasonableness, in that they should be considered reasonable by people who are willing to propose fair terms of cooperation and abide by them provided that others do. In this sense, institutions like NICE reflect a duty of reciprocity among the beneficiaries of a scheme of shared savings: those who benefit from scheme of mutual advantage have an obligation to restrain their demands where this is required for the viability of that scheme.

Given this, it is not surprizing to find that many countries now have institutions like NICE that conduct various forms of health technology assessment and health-care priority setting. Rather, perhaps what is more surprizing is that certain countries have been known to deny the need for such bodies altogether. Indeed, it is a natural outcome of the analysis thus far that while there might be room for reasonable disagreement about some of the particular social or ethical values and the processes that NICE uses, it does not look like there is much room for reasonable disagreement that a system of shared savings in health will require some kind of priority-setting body for health care. In fact, alternatives seem unreasonable. That is, the [End Page 122] only alternatives to authorizing a body to set priorities would appear to be unconstrained expansion of the health-care budget (something that neither governments nor citizens in systems of shared savings should allow) or forms of implicit rationing (which would fail to meet requirements of transparency and consistency). Neither approach is reasonable. Society is best served if it has institutions and agencies in which the terms of the social contract for shared health savings can be clarified and openly debated.

It is difficult to deny, then, that it is reasonable to have a body that performs health-care prioritization in a system of shared savings. But it does not yet follow from this that the particular values and processes through which NICE has attempted to deal with these problems are reasonable. It is to this matter that we turn in the next two sections.

4. THE REASONABLENESS OF NICE’S APPROACH

As seen in Section 2, NICE uses cost effectiveness as a “filter”: a cost per QALY of less than the threshold amount will nearly always be sufficient to guarantee that the medical technology is approved. Above the threshold, a technology will be approved if there are social and ethical values that give sufficient reason to overturn the default assumption. There are two fundamental ways, then, in which the reasonableness of NICE’s methods need to be justified: a) the reasonableness of having a cost-effectiveness threshold that functions as a default; and b) the reasonableness of departing from that default on the basis of a given set of social and ethical values. We turn to related matters, such as the justifiability of NICE’s actual choice of cost-effectiveness threshold, in Section 5.

a. The Reasonableness of NICE’s Use of a Cost-Effectiveness Threshold as a Default

The main justification for NICE’s focus on cost effectiveness again derives from the ethics of opportunity costs. Section 3 explained how attention to opportunity costs is required in order to protect a system of shared savings. Given that money spent on one set of interventions inevitably displaces resources devoted to other interventions, public authorities need to evaluate the opportunity costs involved in funding one set of health-care interventions over other. Within such a protocol, NICE’s attention to cost effectiveness is rendered as an implicit value judgment on the acceptability of certain displacements over others: that is, to be considered acceptable, those technologies that displace existing services must be able to secure a certain level of health benefit, relative to those [End Page 123] services it displaces. The recognition of this logic is explicitly recognized by NICE when it states that those “developing clinical guidelines, technology appraisals or public health guidance must take into account the relative costs and benefits of interventions (their ‘cost effectiveness’) when deciding whether or not to recommend them” (NICE 2008). Such logic might also be seen to follow Ronald Dworkin (2000, 315), who once argued that it would be a “disservice to justice” for some citizens to expect others to contribute to cost-ineffective care under resource constraints, that is to say care that only marginally extends life of relatively poor quality.

Again, there may be disagreement about whether NICE’s privileging of certain technologies on the basis of their ability to secure a certain level of health benefit over those that do not is the most important concern in questions of health-care justice. However, to the extent that such disagreement exists, it looks like reasonable disagreement in the sense we have described: that is, a disagreement that persists even amongst reasonable and well informed individuals despite all the evidence and arguments that can be brought to bear on both sides. In this sense, then, while commentators might question whether NICE’s use of a cost-effectiveness threshold as a default represents the optimum strategy in priority-setting dilemmas, they would be hard pressed to argue that such a strategy is unreasonable.

b. The Reasonableness of Departing from the Default on the Basis of a Given Set of Social Values

As set out in Section 2, although NICE’s method gives primacy to cost effectiveness, it does not make decisions solely on whether a given intervention passes its threshold. Rather it recognizes that other social or ethical values act as constraints on prioritizing cost effectiveness, and that these values can justify exceeding the general cost per QALY threshold. Again, this is eminently reasonable.

If NICE’s focus on cost effectiveness reflects a shared interest in the prudent use of resources, it is important that that interest is also framed by sensitivity to other claims that individuals have as a matter of justice to the benefits of shared savings for health care over and above cost effectiveness. An integral element in the Rawlsian conception of reasonableness is that any social institution is limited in the values that it can admit (Rawls 1996, 57). Any policy paradigm, like the default satisficing strategy of NICE, will therefore bias decision making to some values and not others, a possibility that can be empirically demonstrated (Landwehr and Klinnert 2015). If [End Page 124] the cost-effectiveness threshold were not capable of being overturned by other considerations, so that only the prudential interest in costworthy health care were recognized, then NICE would in effect be making a closed world assumption. It would be saying that its basic principles covered all possible cases. However, a rebuttable presumption has a distinct logic from a closed world assumption, because the presumption is defeasible. Rather than assuming that justified priority-setting is logically deducible from a fully adequate set of principles, NICE’s modifiers allow for a broad and open range of considerations to be advanced, so preserving the reasonableness of its decision-making protocols. It is for the same reason that NICE’s modifiers are preferable to numerically weighted QALYs (reference), given that they preserve the exercise of judgment within the realm of reasonable disagreement.

5. AREAS FOR IMPROVEMENT

We have argued that the general structure of NICE’s method—its setting of a presumption to prioritize cost effectiveness that is rebuttable in light of certain social or ethical values—is reasonable. But to say that this general structure is reasonable is not to say that the structure, or the particular way in which NICE applies it, cannot be improved. In the following, we outline three areas in which NICE’s application of its method looks questionable. We also argue that each of these might be seen as a problem of procedure—of NICE failing to make its reasoning transparent—rather than of substance—of NICE committing an error, or making arbitrary decisions. However, absent proper explanation, NICE remains open to the charge that its actions are, in some respects at least, unreasonable.

a. The Cost-Effectiveness Threshold

The reasonableness of NICE’s cost-effectiveness threshold depends not just on the fact that it sets a cost-effectiveness threshold but on the level at which that threshold is set and the rationale for setting it at this level.

As set out in Section 2, despite NICE’s explicit assertions to the contrary, analysis of its decisions seems to imply that it currently operates a basic cost-effectiveness threshold of around £20,000 to £30,000 per QALY gained (i.e. prior to the influence of any other considerations). Although NICE also remains silent on the rationale for this threshold (which it has not explicitly claimed exists), other sources make clear that it is taken to be the mean cost of producing a QALY elsewhere in the health-care system (Claxton and Culyer 2007). If £20,000 to £30,000 is the mean cost of [End Page 125] producing a QALY within the NHS, then this threshold looks reasonable: within a fixed budget system, funding a new treatment will have the implication that some activities currently being undertaken will have to be foregone. If a new treatment is funded that costs more per QALY than the mean costs of producing a QALY, then funding the new treatment will displace activities that would have created more QALYs. So, unless there are social or ethical values that support funding the treatment despite its displacing a greater health benefit elsewhere, it should not be funded.

However, NICE’s use of the £20,000–£30,000 threshold looks questionable on a number of points. First, we might be generally suspicious of the thought that this particular threshold marks the mean cost of producing a QALY within the NHS, simply on the basis that it does not appear to have been altered in NICE’s lifetime. Even assuming that the threshold was set at the right level when NICE began 15 years ago, then, it would be very surprizing if it were still at the right level now. Second, recent research has suggested that there is good reason to doubt whether £20,000–£30,000 is the mean cost of producing a QALY within the NHS. Claxton et al. (2013), for example, make a persuasive case that the cost to QALY in the wider NHS being £12,936 per QALY gained, meaning NICE’s existing threshold is too high and that recommended interventions are displacing those purchasing greater numbers of QALYs at equal or lower cost.

b. NICE’s Choice of Social or Ethical Values

NICE’s choice of certain social or ethical values as considerations with the power to overturn its cost-effectiveness threshold might be considered a paradigmatic case of a set of judgments about which people reasonably disagree. Paraphrasing Rawls, it seems we might reasonably question NICE’s choice of certain values as either relevant or irrelevant, its formulation of those values, as well as the relative weight it assigns to included values within its decision-making process (both in respect to its more general priority of cost effectiveness and in relation to one another).

However, even if we recognise the outcome of these kinds of decisions as inherently contentious, the way in which NICE choses which values to include and which to exclude still needs to abide by requirements of good public reasoning in order to be considered reasonable. That is, NICE’s choices still need to be made on the basis of reasons that are accepted as relevant by people who are trying to find ways of cooperating with each other on mutually acceptable terms (Rawls 1996). The way in which [End Page 126] NICE seems to choose which social and ethical values might overturn its cost-effectiveness threshold, however, remains confused and confusing. In Social Value Judgements, for example, NICE sets out a range of reasons for rejecting a range of different social or ethical values but such rationales lack consistency and clarity: the “rule of rescue” is rejected on normative grounds (the need to balance the needs of individual patients with the needs of present and future users of the NHS); “age as a proxy for social worth” and “personal responsibility” are rejected because the Citizen’s Council said so; “rare conditions” are excluded because they fall outside NICE’s remit (other NHS institutions manage the availability of “ultra-orphan drugs”). On occasion, values are also rejected without explanation: for example, no explicit reason is given for why “socio-economic status” should not be considered, though it is explicitly ruled out.

To better defend the reasonableness of its choices, NICE needs a clear process for deciding on which social and ethical values can overturn its cost-effectiveness threshold and which are deemed irrelevant. Most notably, it needs to find a way of arbitrating between the different demands of ethical considerations (for example, its considerations of normative arguments in the case of its exclusion of “the rule of rescue”), procedural dictates (for example, its focus on the outcome of Citizen Council meetings in its exclusion of “personal responsibility”) and institutional directives (for example, its considerations of NICE’s remit in its exclusion of “rare conditions”).

c. How NICE Uses Social and Ethical Values in its Decision-Making Process

As we have argued, NICE’s use of modifiers is central to the reasonableness of its methods. However, in introducing such a logic of defeasibility, NICE potentially opens itself up to another charge, namely that is specifies no explicit methods as to how the modifications are to be undertaken or what would be a maximal weight for individual values or all in tandem (although, as mentioned earlier, precedence seems to discourage a maximum cumulative weight of over 2.5 from a starting point of £20,000 per QALY) (NICE 2013).

NICE’s process of applying the social and ethical values outlined in its Social Value Judgements is, if anything, less clear. Social Value Judgements (2008) explains that these values should be used by NICE and its Appraisal Committees “in designing the processes it uses to develop its guidance (recommendations), and in developing individual pieces of guidance.” [End Page 127] However, at no point does the document explain how such Committees are supposed to employ the principles it outlines; for example, whether they are supposed to consider the impact of each and every value in each technology appraisal, how the different demands of each value are supposed to be balanced against one another, how they are supposed to relate to “modifiers” such as certainty and innovation, and so on. At one point, Social Value Judgements does instruct Committees to include a statement of “broad compliance” with the principles it outlines, and justify any departure from the principles it includes. However, this instruction tells Committees only that they must not act against the social and ethical values detailed in the document, not how to bring such principles to bear on the judgments they are making.

At this point, of course, it may be argued that no one has yet constructed an algorithmic method to balance various values against one another and that, as such, to criticise NICE for failing to succeed on this score is to hold them to an unachievable ideal. However, although it is true that such a method is lacking, various proxies are available, not least arriving at a judgement through deliberation. Indeed, it may be that this is the method that NICE uses. Yet, if NICE relies on deliberation, then it really needs to say so. For without such an explanation, NICE leaves the way in which it uses social and ethical values open to criticism.

d. A Substantive Issue or a Process Issue?

Following from the last point, it could be argued that all the criticisms levelled at NICE above are open to an important counterargument: namely, that such criticisms mistake a problem of process for one of substance. On this line of argument, then, it is not that NICE’s methods, either in their setting of the threshold, choosing of social and ethical values, or taking those social and ethical values into account in their decisions, are themselves deficient, rather it is just that NICE has failed to adequately explain and articulate its various ways of working and rationales for the decisions it makes.

Given that we are commenting on NICE’s processes from the outside, this possibility must be recognized. However, even if NICE’s failure is only a failure to adequately communicate its processes, that is still a significant criticism of its ways of working. As commented in the introduction to this paper, priority-setting bodies such as NICE may be evaluated not just on the principles they bring to bear in their decision-making processes (as we have done in this article), but on the character of their processes. In the [End Page 128] latter category, the most influential model of good practice here is AfR, which claims that priority-setting bodies such as NICE need to meet four conditions to be considered fair: that decisions regarding coverage for new technologies and their rationales must be publicly accessible; that these rationales must rest on evidence, reasons, and principles that all fair-minded people can agree are relevant; that there is a mechanism for challenge and dispute resolution regarding limit-setting decisions; and that there is either voluntary or public regulation of the process to ensure that the first three conditions are met (Daniels and Sabin 2008; Daniels 2008). Generally speaking, NICE fares well when measured against AfR’s four conditions of a fair decision-making procedure. Indeed, NICE has added several important procedural requirements missing from the original AfR framework (for example, stakeholder involvement through its use of Citizens Councils for revising its processes and, in particular, social value judgements). However, if NICE has failed to adequately communicate its processes around setting the threshold, choosing social values, or using social values to justify departures from the threshold, it has also failed to abide by best practice described by AfR, namely by failing to make the rationales for its decisions publicly available.

6. CONCLUSION

NICE’s approach to priority setting provides a general framework in which the worth of different interventions can be discussed. What NICE’s process offers is a priority-setting process that privileges the value of cost effectiveness but considers it defeasible in light of other social or ethical values. It operates, then, by a presumptive decision framework, or default approach, that clarifies issues and enables citizens and their representatives to see what complexities emerge. In this, we consider NICE’s approach eminently reasonable, even if it is an approach that can meet with reasonable disagreement.

Benedict Rumbold

Benedict Rumbold is a Research Fellow in the Department of Philosophy at University College London. Benedict’s research concerns ethical and political questions at various levels of abstraction. He holds a PhD in Philosophy from Birkbeck College, University of London.

Albert Weale

Albert Weale is Emeritus Professor of Political Theory and Public Policy at University College London. His research and publications have concentrated on issues of political theory and public policy, especially the theory of justice and the theory of democracy, health policy and comparative environmental policy. He has published widely in health policy, and his latest book is Democratic Justice and the Social Contract (Oxford University Press, 2013).

Annette Rid

Annette Rid is a Senior Lecturer in Bioethics and Society in the Department of Global Health & Social Medicine, King’s College London, and a Fellow of the Hastings Center. Trained in medicine, philosophy and bioethics in Germany, France, Switzerland and the United States, Annette’s research interests span research ethics, clinical ethics and justice in health and health care.

James Wilson

James Wilson is a Senior Lecturer in the Department of Philosophy at University College London. James’s research integrates philosophy with other relevant disciplines, such as epidemiology, economics and political theory, to explore conceptual and practical challenges in the sustainable and equitable improvement of human wellbeing. He focuses particularly on public health ethics, and the ownership and governance of ideas and information.

Peter Littlejohns

Peter Littlejohns is Professor of Public Health in the Faculty of Life Sciences and Medicine at King’s College London and Honorary Consultant to Guy’s and St Thomas’ NHS Foundation Trust. From 1999 to 2012 he was the founding Clinical and Public Health Director of the National Institute for Health and Care Excellence (NICE). He left the Institute to establish an international multidisciplinary collaboration which undertakes research in how to prioritize health care in a fair and equitable way.

ACKNOWLEDGEMENTS

Annette Rid received funding from the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme (FP7/2007-2013) under REA grant agreement no. 301816.

Peter Littlejohns received funding from the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South London at King’s College Hospital NHS Foundation Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health.2 [End Page 129]

NOTES

1. These include clinical guidelines, which advise on the appropriate treatment and care of patients with specific diseases and conditions; guidance on interventional procedures, which detail the safety and efficacy of any surgery, test, or treatment that involves entering the body; and guidance on public health activities, i.e. those that aim to promote a healthy lifestyle and prevent ill health (for example, giving advice to encourage exercise or providing support to encourage mothers to breastfeed).

2. An important note on terminology here: the suggested distinction here between NICE’s assessment of cost effectiveness and its use of “social values” as grounds upon which decisions may depart from a strict adherence to that assessment can be misleading. For, of course, NICE’s use of cost-effectiveness data and the metrics upon which it is based are themselves imbued with social value judgments; for example, one might think of Dan Brock’s seminal work on the value judgments implicit in the use of the QALY (Brock 2004). Principle 3 of NICE’s Social Value Judgments is typical of this confusion, since it appears to draw a contrast between decisions based on evidence of a technology’s “relative costs and benefits” and decisions based on an understanding of how health resources can be distributed “fairly.” However, in order to ensure readability across NICE documents and subsequent commentary, we maintain this somewhat artificial divide over the course of this paper.

REFERENCES

Anderson, Jeffrey L., Paul A. Heidenreich, Paul G. Barnett, et al. 2014. “ACC/AHA Statement on Cost/Value Methodology in Clinical Practice Guidelines and Performance Measures: A Report of the American College of Cardiology/American Heart Association Task Force on Performance Measures and Task Force on Practice Guidelines.” Circulation 129: 2329–45.
Braybrooke, David, and Charles E. Lindblom. 1963. A Strategy of Decision: Policy Evaluation as a Social Process. London: Collier-Macmillan.
Brock, Dan. 2004. “Ethical Issues in the Use of Cost Effectiveness Analysis for the Prioritization of Health Care Resources.” In Bioethics: A Philosophical Overview. Edited by George Khusfh. Dordrecht: Kluwer Publishers.
Cardiosource Website. 2014. “ACC/AHA to Include Cost/Value Assessments in Guidelines and Performance Measures.” Accessed September 8, 2014. http://www.cardiosource.org/en/News-Media/Publications/Cardiology-Magazine/2014/03/ACCAHA-to-Include-CostValue-Assessments-in-Guidelines-and-Performance-Measures.aspx. [End Page 130]
Claxton, Karl, and Anthony J. Culyer. 2007. “Rights, Responsibilities and NICE: A Rejoinder to Harris.” Journal of Medical Ethics 33: 462–64.
Claxton, Karl, Steve Martin, Marta Soares, et al. 2013. Methods for the Estimation of the NICE Cost Effectiveness Threshold. Centre for Health Economics, University of York. Accessed September 8, 2014. https://www.york.ac.uk/media/che/documents/papers/researchpapers/CHERP81_methods_estimation_NICE_costeffectiveness_threshold_(Nov2013).pdf.
Culyer, Anthony J. 2006. “NICE’s Use of Cost Effectiveness as an Exemplar of a Deliberative Process.” Journal of Health Economics, Policy and Law 1 (3): 299–318.
Daniels, Norman. 1994. “Four Unsolved Rationing Problems: A Challenge.” The Hastings Center Report 24: 27–29.
———. 2008. Just Health: Meeting Health Needs Fairly. Cambridge: Cambridge University Press.
Daniels, Norman, and James E. Sabin. 2008. Setting Limits Fairly: Learning to Share Resources for Health. Second Edition. Oxford: Oxford University Press.
Department of Health. 2013. NHS Constitution. London: Department of Health. Accessed September 8, 2014. http://www.nhs.uk/choiceintheNHS/Rightsandpledges/NHSConstitution/Documents/2013/the-nhs-constitution-for-england-2013.pdf.
Devlin, Nancy, and David Parkin. 2004. “Does NICE Have a Cost-Effectiveness Threshold and What Other Factors Influence its Decisions? A Binary Choice Analysis” Health Economics 13 (5): 437–52.
Dworkin, Ronald. 2000 Sovereign Virtue. London: Harvard University Press.
Glassman, Amanda, and Kalipso Chalkidou. 2012. Priority-Setting in Health: Building Institutions for Smarter Public Spending. Washington, D.C., Center for Global Development. Accessed September 8, 2014. http://www.cgdev.org/sites/default/files/1426240_file_priority_setting_global_health_FINAL_0.pdf.
Gutmann, Amy, and Dennis Thompson. 2004. Why Deliberative Democracy? Princeton, N.J.: Princeton University Press.
Hill, Suzanne, Silvio Garattini, Jos van Loenhout, et al. 2003. Technology Appraisal Programme of the National Institute for Clinical Excellence: A Review by WHO. Copenhagen: WHO Regional Office for Europe.
House of Commons Health Committee. 2007. National Institute of Health and Clinical Excellence, First Report of Session, Volume 1. London: The Stationery Office.
Landwehr, Claudia, and Dorothea Klinnert. 2015. “Value Congruence in Health Care Priority Setting: Social Values, Institutions and Decisions in Three Countries.” Health Economics, Policy and Law 10(2): 113–32. [End Page 131]
Littlejohns, Peter, and Michael Rawlins, editors. 2009. Patients, the Public and Priorities in Healthcare. Oxford and New York: Radcliffe Publishing.
Littlejohns, Peter, Tarang Sharma, and Kim Jeong. 2012. “Social Values and Health Priority Setting in England: ‘Values’-Based Decision Making.” Journal of health Organization and Management 26(3): 363–73.
Lord, Joanne, George Laking, and Alastair Fischer. 2004. “Health Care Resource Allocation: Is the Threshold Rule Good Enough?” Journal of Health Services Research and Policy 9 (4): 237–45.
Marmor, Theodore R., and Rudolf Klein. 2012. Politics, Health, and Health Care. New Haven and London: Yale University Press.
Neumann, Peter J., and Dan Greenberg. 2009. “Is the United States Ready for QALYs?” Health Affairs 28 (5): 1366–71.
NICE Website. 2014a. “NICE International.” Accessed September 8, 2014. http://www.nice.org.uk/about/what-we-do/nice-international.
———. 2014b “The Citizens Council.” Accessed September 8, 2014. http://www.nice.org.uk/aboutnice/howwework/citizenscouncil/citizens_council.jsp.
———. 2014c. “Citizens Council Reports.” Accessed September 8, 2014 http://www.nice.org.uk/aboutnice/howwework/citizenscouncil/reports.jsp.
NICE. 2002. Health Select Committee Inquiry: Submission of Supplementary Evidence. Part 2 of 2.
———. 2005. Social Value Judgments: Principles for the development of NICE guidance. London: NICE.
———. 2006. Statins for the Prevention of Cardiovascular Events: Technology Appraisal 94. Accessed September 8, 2014. http://www.nice.org.uk/nicemedia/pdf/TA094guidance.pdf.
———. 2008. Social Value Judgments: Principles for the Development of NICE Guidance, Second Edition. London: NICE. Accessed September 8, 2014. http://www.nice.org.uk/media/C18/30/SVJ2PUBLICATION2008.pdf.
———. 2013. Guide to the Methods of Technology Appraisal 2013. London, NICE. Accessed September 8, 2014. http://www.nice.org.uk/media/D45/1E/GuideToMethodsTechnologyAppraisal2013.pdf.
———. 2014. Value Based Assessment of Health Technologies: Consultation Paper. Accessed September 8, 2014. http://www.nice.org.uk/getinvolved/currentniceconsultations/MethodsOfTechnologyAppraisalConsultation2014.jsp?domedia=1&mid=FE20429F-FA31-FE7B-0F83F51F35753970.
Pollack, Andrew. 2014. “Cost of Treatment May Influence Doctors.” The New York Times, April 17. Accessed September 8, 2014. http://www.nytimes.com/2014/04/18/business/treatment-cost-could-influence-doctors-advice.html?smid=tw-nytimeshealth&seid=auto&_r=1. [End Page 132]
Rawlins, Michael, and Andrew Dillon. 2005. “NICE discrimination.” Journal of Medical Ethics 31: 683–84.
Rawlins, Michael, David Barnett, and Andrew Stevens. 2010. “Pharmacoeconomics: NICE’s Approach to Decision-Making.” British Journal of Clinical Pharmacology 70 (3): 346–49.
Rawls, John. 1996. Political Liberalism. New York: Columbia University Press.
Rawls, John. 2001. Justice as Fairness: A Restatement. Cambridge, Massachusetts: Harvard University Press.
Rid, Annette. 2009. “Justice and Procedure: How Does ‘Accountability for Reasonableness’ Result in Fair Limit-Setting Decisions?” Journal of Medical Ethics 35 (1): 12–16.
Schlander, Michael. 2008. “The Use of Cost-Effectiveness by the National Institute for Health and Clinical Excellence (NICE): No(t Yet an) Exemplar of a Deliberative Process.” Journal of Medical Ethics 34 (7): 534–39.
Shah, Koonal, Cecile Praet, Nancy Devlin, et al. 2012. “Is the Aim of the English Health Care System to Maximize QALYs?” Journal of Health Services Research and Policy 17 (3): 157–63.
Simon, Herbert A. 1976. Administrative Behavior: A Study of Decision-Making Processes in Administrative Organizations. Third Edition. New York: The Free Press.
———. 1983. Reason in Human Affairs. Oxford: Basil Blackwell.
Social Values Project Website. 2014a. “Novo Seven Case Study.” Accessed September 8, 2014. http://www.ucl.ac.uk/socialvalues/case-studies/downloads/Novo_Seven.
———. 2014b. “Sunitinib Case Study.” Accessed September 8, 2014. http://www.ucl.ac.uk/socialvalues/case-studies/downloads/Sunitinib.
———. 2014c. “Trastuzumab Case Study.” Accessed September 8, 2014. http://www.ucl.ac.uk/socialvalues/case-studies/downloads/Trastuzumab.
———. 2014d. “Lapatanib Case Study.” Accessed September 8, 2014. http://www.ucl.ac.uk/socialvalues/case-studies/downloads/Lapatanib.
———. 2014e. “Lucentis Case Study.” Accessed September 8, 2014. http://www.ucl.ac.uk/socialvalues/case-studies/downloads/LUCENTIS.pdf.
Tengs, Tammy O., Gregg Meyer, Joanna E. Siegel, Joseph S. Pliskin, John D. Graham, and Milton C. Weinstein. 1996. “Oregon’s Medicaid Ranking and Cost-Effectiveness: Is There Any Relationship?” Medical Decision Making 16 (2): 99–107.
The Health and Social Care Information Centre. 2010. Prescriptions Dispensed in the Community: England, Statistics for 1999 to 2009. Accessed September 8, 2014. http://www.hscic.gov.uk/catalogue/PUB01428/pres-disp-com-eng-1999-2009-rep.pdf. [End Page 133]
Towse, Adrian. 2002. “What is NICE’s Threshold? An External View.” In Cost Effectiveness and Thresholds: Economic and Ethical Issues. Edited by Nancy Devlin and Adrian Towse. London: Kings Fund/Office for Health Economics.
White, Joseph. 1995. Competing Solutions: American Health Care Proposals and International Experience. Washington DC: The Brookings Institution.
Williams, Alan. 2004. What Could Be Nicer than NICE? London: Office of Health Economics.
World Health Organisation. 2014. Making Fair Choices on the Path to Universal Health Coverage: Final Report of the WHO Consultative Group on Equity and Universal Health Coverage. Geneva: World Health Organisation.