Amgen Response to ICER’s Call for Comments on its Value Assessment Framework

In response to ICER's "Call for Comments on its Value Assessment Framework," Amgen directs ICER to the extensive comments that we have provided over the past 14 months in response to other ICER comment documents, through ICER/manufacturer calls and our public comments at the New England CEPAC (September 2015) and Mid-West CEPAC (May 2016) meetings. ICER does not explicitly state that it endeavors to employ rigorous, transparent, and completely objective scientific methods in its assessments, and our experience suggests that these principles are substantially lacking in ICER's framework, as highlighted below:

• Full Transparency: We have been unsuccessful in our efforts to replicate ICER's model results.  The ICER models remain black boxes that cannot be replicated by the groups ICER thinks should consume and consider them.  Although ICER has made some effort to share high-level model specs, the process remains far from fully transparent. 
  
  Recommendation: Make research methods, assumptions, data inputs, and equations available in a completely transparent manner, such that results are fully reproducible by third parties.^1-3  This is a critical first step towards becoming a trusted organization.
   
• Patient Centricity: ICER does not protect patient interests by paying special attention to data that patients can uniquely provide.  In addition, ICER does not use studies on patient Willingness to Pay (WTP) to inform its thresholds.
  
  Recommendation: Keep the patient at the center of the analysis by having a lower threshold for incorporating data derived directly from patients, such as survey data and health related quality of life studies, using effects that are more meaningful to patients, and performing extensive sensitivity analyses that invoke common sense patient benefits which may have been omitted or impossible to collect in clinical trials.^1-3,5,6
   
• Flexible Willingness to Pay Thresholds:  ICER's current assessments apply the threshold range of $100,000 to $150,000 per quality-adjusted life year (QALY).  A "one-size-fits-all" cost-per-QALY threshold is known to be inherently biased against the oldest and sickest patients, as well as those with the rarest diseases.  Further, ICER applies a cost-per-QALY threshold that has not been validated in the US, much less shown appropriate across every disease, patient group, and medical situation.
  
  Recommendation: Inform thresholds with data, as it is available (e.g., literature-based QALY estimates for cancer treatment in the US).  The cost-per-QALY thresholds should be flexible and appropriate to the society and the condition being treated.^2,3,6 Special considerations such as upwardly skewed age distribution, excessive discounting of life years based on sicker patients, or orphan disease status must be included in the analysis.
   
• Budget Impact is Not "Value": ICER positions its "Potential Health System Budget Impact (Short-Term)" as an assessment that directly informs "Provisional Health System Value".  Although ICER has slightly modified its framework regarding the place of budget impact, the framework still indicates that ICER considers budget impact as the main determinant of health system value. Budget impact analyses poorly reflect value as they treat healthcare spending as a consumption rather than an investment, and do not take into account any long-term health benefits, cost savings, or improved productivity.⁸ 
  
  Recommendation: Remove budget impact as a key driver of the conversation around "Health System Value" and make it clear that budget impact is a completely separate construct from value and not a measure of value.  Instead of budget impact, ICER should provide its Policy Roundtable with a more multi-factorial view that comprise "value" as an investment that provides long-term returns. 
   
• Valid Budget Impact Methodology: ICER's budget impact methods utilize arbitrary caps, exaggerated adoption rates, and focus on short-term time horizons (1-year and 5-year).  Arbitrary budget caps place an inflexible threshold on healthcare spending based on the status quo, and short-term time horizons do not take into account any long-term health benefits.⁸ 
  
  Recommendation: Use non-judgmental budget impact methodology that avoids arbitrary budget caps and reports objective findings.^1,2,5,6  ICER should also (1) utilize realistic adoption rates using real-world adoption rates from reasonable analogs, and perform extensive sensitivity analysis; (2) use real costs, not artificial prices; and (3) focus on multiple time horizons, including those that incorporate longer-term offsets due to patent expiration.^1,5
   
• Provision of Context and Uncertainty: ICER simplifies its answers to easily misunderstood absolutes and averages, without context or quantification of uncertainty. 
  
  Recommendation: Present results as ranges, with extensive sensitivity analyses, rather than absolutes based on average treatment effect.
   
• Contextual Considerations: ICER buries important determinates of value into "Contextual Considerations" where they have much less visibility, do not influence the quantitative analysis, and are never reported. 
  
  Recommendation: Include varied and flexible valuation methods in the framework that synthesize the value from all of the areas ICER currently recognizes as important but fails to formally consider in the value analysis.^1,2
   
• Relevance to Clinical Practice: ICER does not consider less "convenient" sources of data that may be more externally valid (i.e., representative of real-world situations in clinical practice) such as real-world evidence, registries, actual price vs list price, etc. 
  
  Recommendation: Ensure full consideration and use of data that may be more generalizable to clinical practice (e.g., more generous use of and validation against real-world data).^1-5  ICER should also run more extensive alternative analyses (e.g., probabilistic sensitivity analyses) with such data and collect new data as needed to best address the decision problem being considered.  Similarly, the choice of comparators, place in therapy, and utilization assumptions need to be informed by real-world data and extensive input from experts in the field.^3,5
   
• Multi-Stakeholder Perspective (Expert Opinion): ICER's mission indicates that its process supports "a broader dialogue on value in which all stakeholders can participate fully."  Unfortunately, this is not the case.  While there has been increased opportunities for manufacturers to engage with ICER, feedback that is provided is often not incorporated or reflected in ICER's output, including expert opinion.^1,2,4,7
  
  Recommendation: Reflect expert input from opinion leaders, patient advocates and manufacturers into models and analyses.  ICER should also include expertise on specific disease conditions and representatives from patient advocacy organizations specific to the conditions on its voting panels for the CTAF, New England CEPAC, and the Mid-West CEPAC.  While ICER has made modest changes to update its Policy Roundtable, it is critical to have appropriate representation on ICER's voting panel that will deliberate on the evidence and vote on the "Care Value" of these important treatments for patients.
   
• Heterogeneity of Treatment Effect: ICER does not adequately consider how heterogeneity of treatment effects and differences between patients regarding the value of outcomes will influence the results. 
  
  Recommendation: Report how the results may differ under scenarios where patients respond differentially to alternative treatment options and value the various outcomes achieved.^1-3

We believe these principles represent best practice for value assessments and ICER should adopt our detailed and focused guidance on these points.  US patients deserve no less when the value and potential access to life-altering therapies is being publically challenged. ICER's current approach to value assessment falls well short of these best practices on many dimensions, and we will continue to remind ICER of the importance of objective and rigorous scientific analyses that must form the basis of any quality improvements in health care. Done in any other way, ICER risks having the opposite effect on the health care system.

References