ICER’s value assessment framework: New process, not so new approach to value

Prescription drug pricing remains a key area of critique within the U.S. healthcare system. In the absence of a central authority that controls the majority of the market, determining the acceptable price of new technologies in the U.S. often is driven by the negotiating prowess of payers and manufacturers. Some have suggested that the U.S. adopt a national regulatory approach, like the National Institute for Health and Care Excellence (NICE) in England, the Institute for Quality and Efficiency in Healthcare (IQWiG) in Germany, and the Canadian Agency for Drugs and Technologies in Health (CADTH) in Canada, which function to provide rigorous evaluations of new technologies and link acceptable price to the “value” of the intervention. 

In recent years, one organization in the U.S., the Institute for Clinical and Economic Review (ICER), has gained increasing visibility in drug pricing and reimbursement conversations. By providing publicly available independent economic assessments of emerging healthcare interventions, self-described as the U.S.’s “independent watchdog on drug pricing,” ICER claims to provide systematic, transparent assessment of the value of healthcare therapies through clinical efficacy assessments, short- and long-term economic modeling, and qualitative considerations that influence overall treatment value. The approach, however, has not been without its critics, and ICER recently sought input on how to reform its methodology.

This paper highlights key adjustments ICER made to its value assessment process in 2020 based on the comments the organization received from a wide range of stakeholders, including manufacturers, academia, patients and payers. Further, this paper explores what these revisions could mean for manufacturers as they generate data to support economic evaluation of their products and approach pricing and reimbursement conversations with key stakeholders in light of ICER value assessment reports.

Since its founding in 2006 by Dr. Steven Pearson, ICER has completed more than 50 value assessments across numerous therapeutic areas including cardiology, dermatology, gastroenterology, neurology, oncology and urology. From an operational perspective, the institute relies on external financial support through donations and grants as well as membership-based programs to support their mission. Although ICER does receive contributions from payers and manufacturers, nearly 80% of their funding is sourced from non-profit donations or government grants and contracts.

ICER has experienced rapid growth in recent years in its ability to conduct value assessments, primarily spurred by the receipt of multimillion-dollar funding from the Laura and John Arnold Foundation. These grants have allowed ICER to expand the number of assessments conducted per year and extend their activities beyond comparative and cost-effectiveness research to include examining justifications for price increases, developing policy recommendations and partnering with other health technology assessment (HTA) organizations outside of the U.S. With such growth, ICER has gotten the attention of various stakeholders and gained traction with managed care organizations. However, it also has drawn ire from some patient advocacy groups and manufacturers surrounding the way the “value” of new interventions are evaluated, resulting price recommendations, and its implication on access. 

ICER’s current value assessment framework is centered around two key domains: long-term value and short-term affordability. To assess long-term value, ICER examines the comparative clinical effectiveness and incremental cost-effectiveness of the new intervention. With the recent revisions, ICER has acknowledged that value cannot be wholly derived from measures of clinical and cost-effectiveness, so contextual considerations and an examination of other benefits and disadvantages are also added into the framework when assessing long-term value. To assess short-term affordability, ICER calculates the potential budget impact of the intervention as a function of price based on the likely percent uptake of the intervention within an eligible patient population. Based on this assessment, it judges whether the anticipated budget impact is within the target thresholds and, if not, it estimates what proportion of the eligible population can still use the intervention to stay within the threshold.

In keeping with its mission of transparency, ICER solicits feedback from various stakeholders for each value assessment conducted—from the pre-announcement to the public meeting stage—in which patient advocacy groups, clinical experts, payers and manufacturers provide suggestions on the scoping of the evaluation and data sources.

In addition, ICER recently requested public input on how to improve their existing value assessment framework by providing a process for organizations to critique the existing methods and comment on draft revisions. The most recent public input informed the publication of the 2020-2023 Value Assessment Framework revisions.

Key revisions to the framework are centered around five key areas: 

1. Expansion of clinical evidence sources: A common critique of past value assessments published by ICER was that the comparative clinical effectiveness analyses excluded data generated outside of randomized clinical trials (RCTs), leading some stakeholders to conclude that ICER had insufficiently analyzed the full scope of how treatments impact patients. To address this, ICER has committed to including real-world evidence (RWE) sources in future assessments. ICER recently announced a partnership with Aetion, a health technology company focusing on real-world data, to identify and, if necessary, generate relevant evidence to complement the existing data sources. This partnership has been immediate, as a recent draft report released by ICER in March 2020 on sickle cell disease incorporated RWE, including incidence rates of acute and chronic conditions and actual costs of acute and chronic events, provided by Aetion’s de novo evidence generation platform to inform cost effectiveness model inputs.¹² It will be of interest to closely monitor how frequently these collaborations occur, which therapeutic areas are targeted, and which evidence domains (such as efficacy, safety, healthcare resource utilization, patient subgroup identification and evaluation) utilize RWE. Another important constraint is the timing of the activities: ICER aims to complete the whole evaluation process within eight or nine months, which means there’s only a short timeframe to complete the RWE generation activities and incorporate the findings into the assessment.
2. Improving patient engagement: ICER has acknowledged the need to more formally include the patient perspective when characterizing the overall value of healthcare interventions and has introduced new procedures to improve patient and patient advocacy involvement. Moving forward, ICER plans to proactively engage patient stakeholders prior to the initiation of assessments and actively work with patient groups to better understand the disease burden and translate patient input into economic models during the evaluation. ICER has also committed to exploring opportunities to gather new data on outcomes of care that are important to patients and provide co-authorship opportunities with patient groups following completion of the assessment to further key policy recommendations. This increased engagement was also implemented in the recent assessment of sickle cell disease treatments where direct patient and caregiver quotes as well as a patient and caregiver survey were used to better understand the humanistic burden of sickle cell disease and to inform cost-effectiveness model inputs for the societal effects of the disease. In addition to specific patient advocacy engagement for assessments, ICER plans to schedule an annual discussion of the emerging pipeline of new treatments for major therapeutic classes to continually obtain patient input on key priorities. The current process suggests that insights gathered from patient engagement may inform how societal perspective is defined or may influence the cost-effectiveness threshold to be used for the main analysis. However, it’s unclear to what extent patient input will influence future value assessment findings or the weight it will carry.
3. New voting structure, revisions to benefits and contextual considerations: ICER has expanded and revised the list of other value domains that will be considered in the overall value assessment. ICER has also modified the voting structure on each of the domains from a dichotomous scale (yes/no) to a three-point scale (lower, intermediate and higher long-term value) to add more granularity to the weight of each additional consideration in the overall value judgement. Within this current approach, “high” impact domains may lead to using the higher end of the cost-effectiveness threshold in estimating the justifiable price. This topic was one of the main criticisms of the patient advocacy groups and manufacturers. It will be interesting to monitor closely to see whether proposed revisions would help address concerns. As these domains are not included in the comparative effectiveness and incremental cost effectiveness analyses formally, and will be considered qualitatively after the health economic evaluation, it’s likely that the influence of these dimensions is likely to be limited and can be inconsistent across different therapeutic areas.
4. Process to reevaluate new evidence after publication of final evidence report: Typically, the evidence used in ICER evaluations for new interventions is what is available at the time of the regulatory approval of the intervention. The data comes from registrational studies and can be limited to fully capture the potential long-term value of the intervention. To address this issue, ICER has committed to reassess compelling evidence that has emerged in the 12 months after a final evidence report has been published. While this is a noteworthy improvement to the process and aligned with what some other HTA agencies do, the guidance document falls short on what constitutes “compelling evidence,” and it doesn’t contain a process as to how potential new evidence could be factored into updates to an existing report. It’s expected that ICER will use discretion when determining how to process additional evidence that may influence the estimated value of the healthcare interventions and whether an update to an existing evidence report is required. When ICER developed and published its “Unsupported Price Increase Assessment” report, one of the criticisms it received was bias in selecting relevant evidence for consideration. Hence, it will, again, require a few real-world examples to better understand whether this promising approach can address concerns.
5. Pilot program for treatments coming from accelerated approval pathway: ICER has introduced an approach that it will pilot to address the challenge associated with an often limited evidence base available at the time of regulatory approval of therapies approved via accelerated pathways (hence the initial assessment). ICER has provided few additional details to date but noted the process will be initiated at the 24-month anniversary of the publication of the final evidence report and the process will span several months. It’s expected that ICER will partner with external organizations, like Aetion, to incorporate generated RWE to inform these updates.

In addition to the key changes to the framework discussed, ICER acknowledges that additional considerations may be needed for ultra-rare, high-impact or curative therapies. While the value framework that ICER uses (cost-effectiveness and budget impact assessment) remain unchanged, ICER has indicated other benefits and contextual factors used in general assessments may carry more weight for ultra-rare and curative therapies, and there may be additional analyses and other unique considerations to include.

One of the most controversial elements of ICER’s value assessment framework–using cost/quality adjusted life years (QALY) as the foundation of the value assessment—has not changed in the 2020 update despite several stakeholders calling for its revision or replacement. Critics have argued that the cost-effectiveness analysis does not adequately assess the value of treatments for subpopulations like the elderly or those with disabilities and move the field away from patient centricity. Instead, industry authorities provided a wide range of recommendations, including:

• Eliminating cost effectiveness from the value assessment framework entirely;
• Using differential cost-effectiveness thresholds (or other flexible frameworks) that are individualized to the severity, burden of illness or patient need to more holistically capture a treatment’s value; and
• Limiting the discretion of the voting panel to determine the weight of specific contextual benefits and considerations, and quantitatively integrating these factors into the assessment results.

ICER has rebutted these suggestions, noting methodological limitations and insufficient consensus on how to quantify individualized treatment considerations into cost-effectiveness thresholds. Further, ICER argues that using a standardized threshold range across all assessments is equitable as assessments are augmented by a discussion of contextual factors and other important considerations that influence value.

ICER continues to highlight that a single cost-effectiveness threshold should not be utilized as the sole factor in determining value, and that the new guidelines aim to provide a more holistic process in determining value. In our view, despite the updates, it’s expected that ICER final evidence reports will yield similar evaluations to those conducted under the previous value assessment framework. It’s unlikely that the adjustments will lead to revolutionary changes in the value assessments, and it won’t likely change the criticisms the organization has received. With increased funding and visibility, we expect that ICER will continue to take the spotlight in the U.S. and be a topic within the broader pricing and reimbursement conversations.

With ICER’s growing capacity to conduct more evaluations using the cost/QALY approach, manufacturers focused on the U.S. market are likely to encounter a steep learning curve in understanding the structure and evolving importance of ICER’s value assessments. While cost-effectiveness evaluations were done and published from the U.S. payer perspective before, manufacturers did not need to engage and work with an organization that conducts such analyses and publishes to broader U.S. healthcare stakeholders. Nevertheless, there’s considerable opportunity—based on ICER’s formal request for engagement on scope and data sources—for manufacturers to provide input to the evaluations. Considering the most recent framework updates, manufacturers that plan strategically and proactively may find ways to help shape the approach, and thus identify and argue for the inclusion of key evidence to ultimately be a partner in value assessments.

While there will be some valuable and transferrable learnings from engagement with more-established HTA authorities, there will also be unique aspects, and the approach will likely be different for each manufacturer and based on the technology and the scope of the evaluation. Given that ICER has no formal authority in pricing or reimbursement in the U.S., even deciding whether or not to engage them is a strategic choice for the manufacturer. It’s also likely that ICER will want to maintain a clearly independent position, thus limiting the closeness of any engagement.

Not with standing these limitations, there’s anecdotal information suggesting that proactive engagement with ICER can lead to more favorable outcomes.[MU1] One recent example is that ICER reevaluated its assumptions and calculations in its draft evidence report for rheumatoid arthritis—in which AbbVie’s Rinvoq was analyzed—in 2019 after receiving stakeholder input that informed a more comprehensive understanding of the disease area and treatment pathway. However, manufacturers have a reason to be wary as a review of assessments conducted prior to 2020 concluded that revisions to ICER’s assessments based on manufacturer input are generally rare. Some comments from manufacturers (around 30%) have led ICER to revise its assumptions, but fewer than 3% of manufacturer comments lead to changes in ICER’s conclusions in the final assessment. It remains unknown how the revisions to the 2020-2023 assessment process and level of stakeholder engagement will impact future reports.

There’s also one recent notable exception: the analyses on the value of remdesivir as a treatment for COVID-19 following regulatory approval. For the initial remdesivir assessment, which was published in a matter of days following approval in May 2020, ICER did not abide by many of the key updates described in the most recent framework. Although the document released was not a formalized report, the model did not incorporate broader societal benefit or other contextual considerations that may have influenced value, nor did ICER include patient perspective or seek manufacturer input/feedback in the process. Unsurprisingly, it also yielded a justifiable price that was considered by many to be extremely low. ICER released an updated report in June 2020 for remdesivir following further review of clinical data, public comment, and input from the manufacturer with largely similar cost-effectiveness and benchmark price ranges as the initial report. The pricing of remdesivir may be an example of the most influential ICER analysis to date, as Gilead chose to price at a point very close to ICER’s recommendation selling to governments of developed countries at $2,340. Interestingly, this is based on an ICER analysis conducted at a threshold of $50,000/QALY and it remains to be seen whether countries that use different QALY thresholds or different pricing systems altogether will be accepting of this price. Additionally, ICER recently announced adaptations to their methods for evaluation of therapies for COVID-19 that outlines modifications that will be made to the value assessment framework for future evaluations for therapies for COVID-19. Most controversially, ICER noted that they may not engage in discussions with all relevant stakeholders prior to the release of an initial report due to the rapid review but will notify manufacturers for an opportunity to provide input. The question now becomes whether this example signals that ICER’s methods are malleable and subject to reversal and if we should expect to see more exceptions moving forward. 

While the influence of ICER on the final pricing and coverage decisions remains to be seen, manufacturers that strategically plan for this engagement with an understanding of ICER’s methods, perspectives, stakeholder involvement and process will be better positioned to inform the process and shape the value conversation. The evidence so far suggests that those who develop a proactive, strategic plan in collaborating with ICER may have more successful engagement versus those who choose to minimize or respond reactively to draft evidence reports.