Immune checkpoint inhibitors (ICIs) have transformed the cancer treatment paradigm.1 2 ICIs can display different response patterns from molecularly targeted and cytotoxic agents.2–4 To address potential differences in response patterns, it may be important to select clinical trial designs, endpoints, and statistical methods that are appropriate for ICIs.1 While overall survival (OS) is the gold standard for defining clinical benefit, the required long follow-up may delay approval of effective treatments for patients with urgent unmet needs.2 5 6 Therefore, there is interest in identifying alternative endpoints that would help predict ICI response and survival benefits earlier, potentially expediting drug approval and patient access.7 Additionally, methods to more effectively capture and characterize tumor shrinkage as a suitable indicator of clinical outcome are needed.8 9

Objective response rate (ORR) is sometimes accepted as an endpoint supporting accelerated drug approvals.1 10 Reduction in tumor size is captured by ORR and considered by the US Food and Drug Administration (FDA) to be a predictor of clinical benefit, potentially allowing for earlier drug approvals than OS.11 Between March 2011 and August 2017, the FDA approved 25 ICI indications in advanced solid cancers, 15 (60%) of which were approved using ORR as the primary endpoint.1 ORR, however, has limitations associated with the categorical nature of Response Evaluation Criteria in Solid Tumors (RECIST) and does not completely characterize the duration (DOR) or depth (DepOR) of a response. While ORR is a direct measure of antitumor activity that can also be used to track changes in tumor burden status over time and provide a comprehensive assessment of disease status,12 ORR alone may not fully characterize ICI responses and benefits.13 DepOR is a more quantitative measure of tumor burden over time.14 Additionally, DepOR only measures the change in the sum of target lesions, but not non-target lesions, from baseline.15 The clinical significance and adequacy of ORR as a surrogate for marketing approval (accelerated and regular) are dependent on other factors, including the DOR, DepOR, and complete response (CR) rates.16 17

The FDA recommends DOR as a key factor when considering the adequacy of ORR to support approval.10 16 As relationships between endpoints may vary based on tumor type and ICI treatment,7 DOR can be tailored for milestone analyses appropriate to the natural history of the tumor type and ICI efficacy. DOR is described using median estimated rates with associated 95% CIs that are compared at the trial level using Kaplan-Meier (KM) plots.18 For patient-level representations, spider plots display individual tumor changes over time relative to baseline burden.19 Traditionally, these data representations of DOR have been based on the subset of responding patients, defined post randomization, and are thus prone to analysis-by-responder bias.20 New approaches to visually and statistically describe DOR in all randomized patients (ITT) are needed to avoid this bias.

Although DOR alone has not been validated as an endpoint supporting drug approval, several ICIs received accelerated approval based on its combination with ORR.21–24 A phase III trial of patients with melanoma receiving an oncolytic virus showed that durable responses (≥6 months) were associated with clinical benefits, including significant OS improvements and clinically meaningful quality of life improvements.25 However, associations between ORR and DOR with OS for immunotherapy have not been established, and DOR neither as an independent measure of clinical benefit nor in combination with ORR has been fully evaluated.

We assessed DOR with nivolumab monotherapy or in combination with ipilimumab in responder and ITT populations using data from phase I, II, and III studies across solid tumors. To explore effective ways to visually represent and statistically analyze DOR, we evaluated the relationship between the combination of ORR and DOR (characterized by the restricted mean time in response (RMTR)) or ORR alone with OS in the ITT population.

We propose a novel approach combining ORR and DOR (RMTR) to objectively and visually characterize tumor shrinkage in the ITT population, which our results suggest correlates with OS and helps capture clinical responses to immunotherapy.

We retrospectively analyzed available patient-level data from completed, large, phase III, active-control, registrational trials conducted by Bristol Myers Squibb (BMS) of nivolumab as monotherapy or in combination with ipilimumab using chemotherapy as a comparator for approved indications in non-small-cell lung cancer (NSCLC), melanoma, renal cell carcinoma (RCC), and squamous cell carcinoma of the head and neck (SCCHN). For further illustration, the DOR analyses also included phase I and II, single-arm, BMS trials from NSCLC (CheckMate 012) and melanoma (CheckMate 069) that were supportive for regulatory submissions and contained the necessary endpoints. Analyses of potential differences in DOR were performed in ITT and responder populations using data from select NSCLC and melanoma trials. We evaluated associations of OS with ORR alone and in combination with DOR, measured by the RMTR, using individual patient data from randomized phase III trials across NSCLC, melanoma, RCC, and SCCHN.

Nine completed phase I, II, and III trials (n=3551) of nivolumab monotherapy or combination therapy with ipilimumab for NSCLC (CheckMate 012, 017, 026, 057, and 063) or melanoma (CheckMate 037, 066, 067, and 069) were selected to study DOR differences in patients treated with ICI therapy compared with chemotherapy (table 1). Associations between ORR and RMTR with OS were evaluated using individual patient data from nine randomized phase III studies (n=5323; table 1) across multiple indications (melanoma: CheckMate 037, 066, and 067; NSCLC: CheckMate 017, 026, and 057; RCC: CheckMate 025 and 214; SCCHN: CheckMate 141).

These results demonstrate how tumor shrinkage can be characterized using DOR or a combination of DOR and ORR (RMTR) in the ITT populations of patients treated with ICIs across multiple tumor types. Combining ORR analyses with DOR can provide a comprehensive measure of tumor responses that cannot be obtained from ORR analyses alone. Nivolumab monotherapy or combination therapy with ipilimumab appears to be associated with a longer DOR versus active control in both responder and ITT populations. To our knowledge, this is the first report of an association between RMTR and OS in patients from an ITT population treated with ICIs. The novel endpoint RMTR may influence clinical trial design and complement traditional endpoints such as OS to capture responses to immunotherapy.

RMTR is a robust, clinically interpretable measure of the average time in response that can be applied to an ITT patient population and used as an alternative measure of treatment effect.43 As an endpoint, RMTR may be an earlier efficacy readout and may capture unique aspects of ICI treatment benefits compared with traditional endpoints. This may impact patient care, as earlier treatment decisions could be made about continuing or switching treatments. Response durability is often measured by comparing DOR between treatment arms among responders.28 Longer maintenance of response associates with lack of disease progression; a reasonable indication that OS should be extended as well. Thus, a novel composite of ORR and DOR could provide more comprehensive information on clinical responses with ICIs than ORR alone.

Advantages of tumor-based endpoints, such as DOR and ORR, compared with time-to-event endpoints, such as OS and PFS, are that the former can be directly attributed to the therapy and might provide an earlier readout of drug activity.13 44 In our study, RMTR and OS were correlated at 6, 12, 18, and 24 months, with similar levels of significance observed at all time points. These preliminary findings suggest that a 6-month cut-off is a reasonable duration of follow-up for DOR/RMTR analyses of ICIs.

ICIs can pose unique challenges for clinical trial design in part related to their mechanisms of action, including heterogeneous mechanisms of primary and acquired resistance, potentially delayed onset of immunologic responses, and optimization of dosing and duration.45 ORR and response duration are important for evaluating efficacy of oncology therapeutics. A measure characterizing both of these responses, such as RMTR, is likely to be a reliable predictor of long-term clinical benefits.

We provide preliminary evidence that RMTR appears to be comparable to ORR alone in its association with survival outcomes across multiple tumor types. However, prospective studies and analyses comparing DOR and RMTR with other endpoints (eg, PFS, durable response rate25) are needed to learn about their potential utility in clinical trials and how these novel endpoints compare with traditional RECIST measures of tumor response. Higher-level evidence is required to extensively establish surrogacy for OS.

In recent years, several approved immunotherapies have demonstrated durable responses,28 46 47 and accelerated approvals have been granted based on ORR.10 However, strong associations between ORR and OS for ICIs have yet to be established. Our identification of a significant association between ORR and OS is novel, and differs from recent meta-analyses, which reported only weak associations in studies of both immunotherapies and targeted therapies.7 44 48–50 Weak correlations in these analyses were likely related to multiple factors, including a crossover study design, patients continuing on additional therapies, long postprogression survival, study heterogeneity (involving sample sizes, disease types and settings, lines of therapy and follow-up durations), and antitumor mechanisms of action associated with ICIs that may contribute to delayed responses or pseudoprogression, which may lead to premature treatment cessation.7 44 48 49

An FDA-sponsored meta-analysis of 14 randomized, active-controlled trials of targeted and standard therapies including 12 567 patients with advanced NSCLC found no association between ORR and OS (R²=0.09; 95% CI 0 to 0.33).44 The authors suggested that ORR alone may not be the optimal endpoint for the expedited approval of cytotoxic therapies or immunotherapies.44 Another meta-analysis of 13 randomized, multicentre, active-controlled trials of immunotherapies submitted for FDA approval found that the association between ORR and OS was weak (R²=0.1277).48 Unlike the data presented here, which used differences in response rates to assess the relative effects of ORR, this previous study used ORs. An inherent challenge with ORs is the potential exaggeration of small differences, a significant issue when response rates are low. In contrast, our approach utilized differences in ORR that have a simpler interpretation based on absolute numbers rather than ratios. Unlike the previously reported study-level meta-analyses, individual patient-level data in this study were analyzed, providing more precision to the association analyses between ORR and DOR with OS. Another possible reason for different results could be that both meta-analyses only included trials submitted for regulatory approval which were positive and thus subject to selection bias, whereas our study included both positive and negative trials.44 48

ORR and RMTR appear to be associated with survival in patients treated with nivolumab monotherapy or with nivolumab plus ipilimumab combination therapy across multiple tumor types. Combining ORR and DOR to evaluate ICI therapy provides a visual, less biased and more informative analysis of the ITT population. Results showed similar associations of this combined endpoint to OS as ORR alone. The novel combination of ORR and DOR (RMTR) is a potentially useful and reliable endpoint for characterizing tumor shrinkage and clinical benefit, that is associated with OS. Our findings provide a strong rationale for further studies needed to establish the potential clinical utility of this endpoint. Further exploration of DOR and RMTR endpoints is warranted, particularly with new therapeutic modalities that have novel mechanisms of action or that involve novel drug combinations involving immunotherapy.