Advertisement

Psychometric Characteristics and Validity of the PROMIS Cancer Function Brief 3D Profile

Open AccessPublished:February 03, 2021DOI:https://doi.org/10.1016/j.apmr.2020.12.027

      Abstract

      Objective

      To develop an item response theory (IRT)–calibrated, patient-reported outcome measure (the PROMIS Cancer Function Brief 3D Profile) of physical function, including associations with fatigue and social participation, in cancer rehabilitation patients.

      Design

      Large-scale field testing, graded response model IRT analyses, and multivariate regression analysis.

      Setting

      Six cancer rehabilitation clinics associated with cancer centers across the United States.

      Participants

      Adults (N=616) treated in outpatient cancer rehabilitation medicine clinics.

      Intervention

      Not applicable.
      Main Outcome Measures: The PROMIS(r) Cancer Function 3D Profile (including existing items from PROMIS(r) item banks).

      Results

      A total of 616 patients completed 21 items in the initial item pool. Nine items were removed because of comparatively lower information that they provide according to the IRT item calibrations, low item-total correlations, or bimodal distributions. The remaining items generated a 12-item short form. Regression analyses determined that the items were responsive to and representative of the patient population across trait ranges and multiple domains and subdomains of function.

      Conclusions

      This psychometric investigation supports the use of the PROMIS Cancer Function Brief 3D Profile for evaluating function in outpatient cancer rehabilitation patients.

      Keywords

      List of abbreviations:

      BMI (body mass index), CAT (computer adaptive testing), CRMMC (Cancer Rehabilitation Medicine Metrics Consortium), ECOG (Eastern Cooperative Oncology Group), IRT (item response theory), KPS (Karnofsky Performance Status), PRO (patient reported outcome), PROMIS (Patient-Reported Outcomes Measurement Information System), SF (short form)
      A significant portion of the estimated 17 million cancer survivors in the United States will experience impaired function as a result of their disease and its treatment.
      • Miller K.D.
      • Siegel R.L.
      • Lin C.C.
      • et al.
      Cancer treatment and survivorship statistics, 2016.
      Although the field of cancer rehabilitation has grown recently, its ability to meet clinical demands lags behind current and projected needs.
      • Silver J.K.
      • Stout N.L.
      • Fu J.B.
      • Pratt-Chapman M.
      • Haylock P.J.
      • Sharma R.
      The state of cancer rehabilitation in the United States.
      • Wu H.S.
      • Harden J.K.
      Symptom burden and quality of life in survivorship: a review of the literature.
      • Holm L.V.
      • Hansen D.G.
      • Johansen C.
      • et al.
      Participation in cancer rehabilitation and unmet needs: a population-based cohort study.
      • Harrison J.D.
      • Young J.M.
      • Price M.A.
      • Butow P.N.
      • Solomon M.J.
      What are the unmet supportive care needs of people with cancer? A systematic review.
      There are many causes for this deficiency, but one that is potentially addressable in the near term is the lack of a scalable and accessible means of assessing physical and social function to align care with patients’ symptoms and goals.
      • Alfano C.M.
      • Mayer D.K.
      • Bhatia S.
      • et al.
      Implementing personalized pathways for cancer follow-up care in the United States: proceedings from an American Cancer Society–American Society of Clinical Oncology summit.
      • Alfano C.M.
      • Mayer D.K.
      • Beckjord E.
      • et al.
      Mending disconnects in cancer care: setting an agenda for research, practice, and policy.
      • Pergolotti M.
      • Alfano C.M.
      • Cernich A.N.
      • et al.
      A health services research agenda to fully integrate cancer rehabilitation into oncology care.
      Accordingly, it is essential to develop psychometrically sound measurement tools that accurately reflect real-world clinical outcomes, are not burdensome to the patient, are cost effective, and do not add measurably to the workload of an already overstretched workforce. Such an instrument could harmonize assessment across multiple institutions and improve the personalization of treatment. Unfortunately, existing measures of function in cancer patients fall short in a number of ways. For example, the PROMIS–Cancer Bank v1.1–Physical Function consists of 45 questions, making it burdensome for patients and of unclear value to providers. Other measures such as the Functional Assessment of Cancer Therapy–General ask patients for information not directly modifiable with rehabilitation, such as nausea and whether family members have accepted a patient’s illness. In addition, many measures do not capture both physical function and related domains and subdomains amenable to rehabilitation, such as fatigue.
      The Cancer Rehabilitation Medicine Metrics Consortium (CRMMC) was formed to address the need for pragmatic, easy-to-administer outcome measures relevant to the delivery of cancer rehabilitation service.
      • Smith S.R.
      • Vargo M.
      • Zucker D.S.
      • Henderson M.
      • Shahpar S.
      • Wisotzky E.M.
      • Custodio C.
      • Basford J.
      • Jay G.
      • Gerber L.
      • Cheville A.
      The Cancer Rehabilitation Medicine Metrics Consortium: A path to enhanced, multi-site outcome assessment to enhance care and demonstrate value.
      The group’s goal was to create an instrument with high applicability across the range of outpatient cancer rehabilitation medicine practices. To meet these criteria, the CRMMC considered extant legacy and item response theory (IRT)-modeled patient reported outcome measures (PROs). Because multiple institutions in the CRMMC were advancing Patient Reported Outcome Measurement Information System (PROMIS)-based monitoring strategies, the decision was made to use PROMIS tools. Advantageous features of PROMIS for use in cancer rehabilitation settings include its applicability to outpatient populations, familiarity among oncology providers, availability in the public domain, and options for either computer adaptive testing (CAT), generic short form (SF), or profiles consisting of multiple SFs.
      However, neither CAT nor existing SF approaches to item selection and administration were deemed an optimal match for the CRMMC’s goals for a number of reasons. Subdomain coverage is not specified in the PROMIS CAT and SF algorithms; thus patients may complete CAT without responding to the items of greatest relevance to cancer rehabilitation providers. The CRMMC therefore elected to create a novel PROMIS measure that included subdomain coverage (eg, fine motor function and fatigue) in trait ranges that are pertinent to the assessment and treatment of disability across cancer populations.
      The CRMMC completed a modified Delphi process to select the domains for assessment and candidate items within each domain. In particular, the physical function, fatigue, and social roles ability domains were chosen given their empirical association with disablement experienced by cancer survivors.
      • Neo J.
      • Fettes L.
      • Gao W.
      • Higginson I.J.
      • Maddocks M.
      Disability in activities of daily living among adults with cancer: a systematic review and meta-analysis.
      • Gerber L.H.
      Cancer-related fatigue: persistent, pervasive, and problematic.
      • Jones J.M.
      • Olson K.
      • Catton P.
      • et al.
      Cancer-related fatigue and associated disability in post-treatment cancer survivors.
      • Brinkman T.M.
      • Recklitis C.J.
      • Michel G.
      • Grootenhuis M.A.
      • Klosky J.L.
      Psychological symptoms, social outcomes, socioeconomic attainment, and health behaviors among survivors of childhood cancer: current state of the literature.
      Participants in this process were the physician authors (SS, MV, DZ, SS, LG, MH, AC) of this manuscript and 2 more from institutions with robust cancer rehabilitation programs but who did not participate in data collection. The IRT item information properties for each bank were reviewed, and the CRMMC identified discriminating items in the target trait ranges. Those believed to be most relevant to the management of disability among patients with cancer were selected, resulting in 21 candidate items for clinical testing.
      Although the Delphi method is an accepted and successful consensus process for developing functional assessment tools,
      • Jobst A.
      • Kirchberger I.
      • Cieza A.
      • Stucki G.
      • Stucki A.
      Content validity of the comprehensive ICF core set for chronic obstructive pulmonary diseases: an international Delphi survey.
      • Herrmann K.H.
      • Kirchberger I.
      • Stucki G.
      • Cieza A.
      The comprehensive ICF core sets for spinal cord injury from the perspective of physical therapists: a worldwide validation study using the Delphi technique.
      • Glocker C.
      • Kirchberger I.
      • Gläßel A.
      • Fincziczki A.
      • Stucki G.
      • Cieza A.
      Content validity of the comprehensive international classification of functioning, disability and health (ICF) core set for low back pain from the perspective of physicians: a Delphi survey.
      the measurement properties of such instruments remain unknowable without clinical testing. This study reports the results of the empirical, multisite process used to field test the 21 candidate items to create a low-burden 12-item assessment tool.

      Methods

      Study sample

      A convenience sample was recruited among outpatient cancer rehabilitation clinics at 6 participating institutions. Inclusion criteria were having a history of cancer and age of 18 years or older. There were no limitations regarding demographic characteristics, disease type, treatment, stage, or duration of survivorship. Patients with cognitive or communication limitations that could interfere with the ability to complete the questionnaire, including lack of English proficiency, were excluded.
      The institutional research boards of each participating site approved the study, and patients were required to provide written consent if required by the Institutional Research Boards where the data were collected. The University of Michigan was the coordinating site and was responsible for monitoring data quality, securely storing the data, and coordinating data use agreements.

      Data collection and management

      Data were collected from patients and their treating physiatrists at initial consultations or follow-up visits. Patients completed the study questionnaire in the waiting area or examination room prior to seeing the physiatrist. Patients were assigned a study identification number and those with multiple clinic appointments during the data collection period were invited to complete the questionnaire at each visit.
      Demographic and clinical data including age, sex, type of cancer, treatment history, and disease stage were abstracted from the participant’s electronic health records and recorded either during or immediately after patient visits. Additional characteristics were recorded given their association with disablement, including body mass index (BMI) and the presence of bone or central nervous system metastases, polyneuropathy, and noncancer comorbidities that might affect function (musculoskeletal, neurologic, cardiopulmonary, psychiatric).
      The 21 items were administered using computer-interfaced tablets or paper forms, depending on site resources. Patient and clinician responses were captured in a REDCapa database.
      • Harris P.A.
      • Taylor R.
      • Thielke R.
      • Payne J.
      • Gonzalez N.
      • Conde J.G.
      Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support.
      ,
      • Harris P.A.
      • Taylor R.
      • Minor B.L.
      • et al.
      The REDCap consortium: building an international community of software partners.
      Investigators who collected responses via paper forms manually entered results into the database; tablet-based responses were automatically recorded. Data were exported as Excelb files and merged in STATA v.15.0c after collection was completed.

      Candidate SF items

      Among the candidate items, 11 assessed physical function with a combination of mobility and activities of daily living questions, 5 assessed fatigue, and 5 assessed ability to participate in social roles and activities items (appendix 1). The items spanned a wide trait range, appeared broadly applicable across diverse cancer populations, and had excellent IRT item information characteristics. Items were selected from the entire PROMIS item bank and not specific SFs; they are named based on existing PROMIS IDs.
      Before inclusion, the reading level of candidate items was analyzed using The Flesch-Kincaid Grade Level formula. In addtion, the approximate metabolic equivalent level of each physical function item was calculated using the Compendium of Physical Activities
      • Ainsworth B.E.
      • Haskell W.L.
      • Herrmann S.D.
      • et al.
      2011 compendium of physical activities: a second update of codes and MET values.
      to confirm that the pool of selected items captured a reasonable range of performance intensity.

      Pain and distress numerical rating scales

      Participants rated their pain and distress on 11-point numeric rating scales. Returning patients also reported a perceived general rating of change item. The distress scale was adapted from the National Comprehensive Cancer Network’s Distress Thermometer.
      • Holland J.C.
      • Bultz B.D.
      The NCCN guideline for distress management: a case for making distress the sixth vital sign.
      The 11-point pain scale is well-validated and ubiquitous in clinical and research settings.

      Clinician-rated outcomes

      Performance status was assessed using the Karnofsky Performance Status (KPS)
      • Schag C.C.
      • Heinrich R.L.
      • Ganz P.A.
      Karnofsky performance status revisited: reliability, validity, and guidelines.
      and Eastern Cooperative Oncology Group (ECOG)
      • Oken M.M.
      • Creech R.H.
      • Tormey D.C.
      • et al.
      Toxicity and response criteria of the Eastern Cooperative Oncology Group.
      scales. Physicians also completed a perceived rate of change item for returning patients.

      Statistical analysis

      Descriptive statistics were used to characterize the study population’s demographic and clinical characteristics. Means and standard deviations (SDs) were calculated for continuous variables and proportions for binary and categorical variables.

      Item information characteristics

      The development and calibration of PROMIS item banks used both general and diseased populations.
      • Lai J.S.
      • Cella D.
      • Choi S.
      • et al.
      How item banks and their application can influence measurement practice in rehabilitation medicine: a PROMIS fatigue item bank example.
      ,
      • Reeve B.B.
      • Hays R.D.
      • Bjorner J.B.
      • et al.
      Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS).
      PROMIS measures are reported on a T score metric (mean, 50; SD, 10) and anchored to the US general population to enable easy interpretation, comparison, and generalizability across normal and diseased populations. The PROMIS item banks were calibrated using the graded response model. We used the item parameter calibrations published from the PROMIS item bank to obtain the item information characteristic curves. In addition, we ran one model for each of the 3 constructs for IRT scoring using published item parameters from the item banks and scored those who responded to at least 50% of the items. Densities of the IRT scores were plotted. The flexMIRT v2.0 softwared was used for the IRT scoring. Item information curves and density plots were compared to characterize trait coverage and informational yield in the target population.

      Anchor-based analyses

      Linear, logistic, and ordinal logistic regression models were constructed to assess which of the candidate items discriminated as expected across anchoring variables. Participants’ item responses were included as indicator variables in the models. Dependent variables included KPS, ECOG, BMI, presence of active cancer, and presence of distant metastases. The regression modeling approach was linear for KPS, ordinal logistic for ECOG and BMI, and logistic for presence of active cancer and distant metastases. Models were adjusted for correlation at the participant-level because multiple participants provided more than one data point. Conventional goodness-of-fit statistics were estimated. All analyses were performed in STATA.

      Final item selection

      The IRT information curves were reviewed for trait coverage and information content. Lesser performing, redundant items, and those with low information content were culled. In a second selection step, CRMMC members reviewed the output from the regression analyses to identify which candidate items strongly associated with related clinical constructs (eg, physical function with KPS). The group also reviewed which items had a stepwise increase in the coefficients of the ordered response options (never [5], rarely [4], sometimes [3], often [2], and always [1]) and which had the broadest coefficient span. The CRMMC weighted the KPS and ECOG models more heavily in decision making because these constructs are more strongly correlated with physical function than BMI or the presence of active disease or metastases. Consensus was reached through group discussion regarding which items to retain in the final assessment tool.

      Results

      Study sample

      A total of 616 unique patients were enrolled across the 6 sites, comprising 1009 assessment points. The PRO typically took patients less than 5 minutes to complete. Patient demographic and clinical characteristics are listed in table 1. Breast cancer was the most common diagnosis (42.5% of the sample), with women comprising two-thirds of the study participants. The average age was 59.4 years (SD, 14.1y). Just over half of the participants had no evidence of disease (56.9%) at the time of evaluation. Nearly all received surgery, chemotherapy, or radiation therapy prior to assessment. Overall, the sample had a high level of physical performance. Only 20% required an assistive device for ambulation and two-thirds had ECOG performance scores of 0 (fully active, able to carry on all predisease performance) or 1 (restricted in physically strenuous activity). Table 2 includes performance site characteristics and breaks down enrollment by cancer diagnosis.
      Table 1Participant characteristics
      Characteristicn (%)
      Percentages may exceed 100% due to multiple diagnoses.
      Age, mean, SD (range), y59.4, 14.1 (21-95)
      Female408 (66.2)
      Visit type
       New387 (62.8)
       Return229 (37.2)
      Cancer management stage
       No evidence of disease308 (50.0)
       Recurrent/palliative163 (26.5)
       Primary treatment145 (23.5)
      Presence of nonregional metastases152 (24.7)
       Bone101 (16.4)
       Brain23 (3.0)
       Spinal cord9 (1.5)
       Leptomeningeal3 (0.6)
      Symptoms of peripheral neuropathy133 (21.6)
      Chemotherapy
       Active157 (25.5)
       Previous324 (52.6)
       Never135 (21.9)
      Hormone associated therapy
       Active130 (21.1)
       Previous64 (10.4)
       Never422 (68.5)
      Immunotherapy
       Active38 (6.2)
       Previous27 (4.4)
       Never551 (89.4)
      Radiation therapy
       Active35 (5.7)
       Previous379 (61.5)
       Never202 (32.8)
      Surgery
      Within 4 weeks19 (3.1)
       >4 weeks previous455 (73.9)
       Never142 (23.1)
      Noncancer musculoskeletal impairment
       None/minimal391 (63.5)
       Moderate193 (31.3)
       Severe32 (5.2)
      Noncancer neurologic impairment
       None/minimal525 (85.2)
       Moderate77 (12.5)
       Severe14 (2.3)
      Noncancer cardiopulmonary impairment
       None/minimal562 (91.2)
       Moderate49 (8.0)
       Severe5 (0.8)
      Noncancer psychiatric impairment
       None/minimal529 (85.9)
       Moderate68 (11.0)
       Severe19 (3.1)
      Body mass index
       <18.510 (1.6)
       18.5-25194 (31.5)
       >25-30190 (30.8)
       >30-35118 (19.2)
       >35-4051 (8.3)
       >40-4550 (8.1)
       Missing3 (0.5)
      Ambulation
       Independent494 (80.2)
       Modified independent96 (15.6)
       Wheelchair26 (4.2)
      KPS
       406 (1.0)
       5019 (3.1)
       6079 (12.8)
       70149 (24.2)
       80165 (26.8)
       90142 (23.1)
       10056 (9.1)
      ECOG
       0 fully active147 (23.9)
       1256 (41.6)
       2159 (25.8)
       350 (8.1)
       4 completely disabled4 (0.6)
      NOTE. N=492.
      Abbreviation: SD, standard deviation.
      Percentages may exceed 100% due to multiple diagnoses.
      Table 2Cancer diagnoses by site
      Cancer TypeSite 1, n (%)
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Site 2, n (%)
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Site 3, n (%)
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Site 4, n (%)
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Site 5, n (%)
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Site 6, n (%)
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Total
      N302130492526124616
      NCI CCC
      ACOS accredited
      Breast100 (35.3)70 (59.3)29 (67.4)12 (50.0)14 (58.3)37 (29.8)262 (42.5)
      Head/neck (nonthyroid)22 (7.8)10 (8.5)1 (2.3)1 (4.2)1 (4.2)22 (17.7)57 (9.3)
      Sarcoma28 (9.9)2 (1.7)02 (8.3)09 (7.3)41 (6.7)
      Brain16 (5.7)9 (7.6)04 (16.7)0029 (4.7)
      Gynecologic11 (3.9)6 (5.1)9 (20.9)2 (8.3)1 (4.2)14 (11.3)43 (7.0)
      Other
      Other cancers do not fit in any other category (eg, esophageal, testicular).
      19 (6.7)4 (3.4)2 (4.7)02 (8.3)027 (4.4)
      Multiple myeloma22 (7.8)3 (2.5)001 (4.2)026 (4.2)
      Prostate12 (4.2)8 (6.8)0009 (7.3)29 (4.7)
      History of allogeneic BMT15 (5.3)4 (3.4)01 (4.2)015 (12.1)35 (5.7)
      Lymphoma (no allogeneic BMT)9 (3.2)6 (5.1)1 (2.3)03 (12.5)019 (3.1)
      Colorectal12 (4.2)1 (0.8)2 (4.7)02 (8.3)4 (3.2)21 (3.4)
      Lung8 (2.8)2 (1.7)3 (7.0)1 (4.2)08 (6.5)22 (3.6)
      Thyroid7 (2.5)2 (1.7)001 (4.2)010 (1.6)
      Melanoma7 (2.5)01 (2.3)1 (4.2)1 (4.2)6 (4.8)16 (2.6)
      Renal7 (2.5)01 (2.3)0008 (1.3)
      Leukemia (no allogeneic BMT)3 (1.1)2 (1.7)01 (4.2)006 (0.9)
      Bladder/ureteral4 (1.4)1 (0.8)00005 (0.8)
      NOTE. N=492.
      Abbreviations: ACOS, American College of Surgeons; BMT, bone marrow transplantation; NCI CCC, National Cancer Institute Comprehensive Cancer Center.
      Percentages represent proportion within site. Exceeds 100% due to multiple diagnoses.
      Other cancers do not fit in any other category (eg, esophageal, testicular).

      IRT item information curves

      We plotted item information curves using the item parameters provided by PROMIS item banks as well as the score density plots (fig 1). Score summary statistics are presented in table 3. For each item, the x-axis of the item information curves shows trait ranges and the y-axis shows the amount of information at each point of the trait. For physical function items (n=1004), PFC45, PFA16, PFA17, and PFB18 had low peak of information (y-axis). PFA23, PFA21, and PFB12 had peak of information of 3 or more and trait range between –2 and 1, indicating that these items captured lower levels of function. PFB24, PFC12, and PFA4 had a trait range between –1 and 2 and were therefore better at capturing higher levels of function.
      Figure thumbnail gr1a
      Fig 1Score distribution of the sample and the item informative curves (ICCs).
      Figure thumbnail gr1b
      Fig 1Score distribution of the sample and the item informative curves (ICCs).
      Figure thumbnail gr1c
      Fig 1Score distribution of the sample and the item informative curves (ICCs).
      Figure thumbnail gr1d
      Fig 1Score distribution of the sample and the item informative curves (ICCs).
      Table 3Summary statistics of the IRT scores
      StatisticsPhysical FunctionFatigueAbility to Participate in Social Roles and Activities
      Minimum–2.95–1.26–2.29
      First quartile–0.640.66–1.20
      Mean (SD)0.00 (0.97)1.26 (0.92)–0.55 (0.90)
      Median–0.031.11–0.65
      Third quartile0.641.86–0.03
      Maximum2.113.121.50
      Abbreviation: SD, standard deviation.
      All 5 fatigue (n=996) items had similar levels of peak information. FATIMP30 and FATEXP22 captured higher levels of fatigue than others, although in general, the items had similar trait ranges and measured severe symptoms better than mild symptoms. Only 5.3% of the respondents scored below the population average.
      Finally, items regarding the ability to participate in social roles and activities (n=982) revealed that 76.2% of respondents scored lower than the population average. All except SRPPER16 had high peak of information levels, and all items had a similar trait range, favoring lower levels of social participation.
      Items were culled based on IRT analysis if they provided a lesser degree of information or if the level of information provided was considerably inconsistent across the trait range (eg, if an item performed poorly at the low end of a trait but better at the high end). The CRMMC prioritized having some items that performed better at higher or lower trait ranges as long as the item did not perform exceedingly poorly at one end. When items provided comparable levels of information and trait coverage, the CRMMC examined their subdomain coverage. Items were prioritized for inclusion if they were relevant to an important subdomain that did not overlap with other items or if the activity described in the stem was clinically relevant. For example, the ability to reach into a high cupboard addresses upper extremity and shoulder function, a highly relevant construct to survivors of breast and head and neck cancers. Items that were presumptively culled on the basis of their IRT information curves included PFB12, PFA17, PFB18, PFC12, FATEXP22, SRPPER16, and SRPPER46. Items potentially fit for inclusion in the final SF based on IRT analysis and clinical judgment, before regression analysis, were PFA23, PFA21, PFB24, PFC45, PFA16, PFA42, PFC12, FATIMP30, FATIMP51, FATIMP18, FATIMP9, SRPPER11, SRPPER18, and SRPPER23.

      Anchor-based assessments and validity

      Review of the regression models was used principally as a “tie breaker” when 2 items offered comparable information and subdomain coverage relevant to cancer survivors. Output from the regression models is listed in table 4.
      Table 4Results of regression analyses
      ItemR2Pseudo R2Prob(F)Response Option Coefficient
      2345
      PFA23—Are you able to go for a walk of at least 15 minutes?
      Items included in the final SF.
      KPS0.440.009.7713.5417.0925.14
      ECOG0.210.00–2.28–2.86–3.84–5.34
      BMI0.020.000.28–0.21–0.44–1.02
      Active Cancer0.030.000.470.450.440.25
      Metastatic0.030.000.770.900.590.34
      PFA21—Are you able to go up and down stairs at a normal pace?
      Items included in the final SF.
      KPS0.370.008.9712.0416.2223.55
      ECOG0.170.00–1.78–2.43–3.36–4.51
      BMI0.240.000.160.13–0.34–1.07
      Active Cancer0.030.000.640.660.560.28
      Metastatic0.030.000.800.950.620.33
      PFB24—Are you able to run a short distance, such as to catch a bus?
      KPS0.350.006.4510.4515.0318.03
      ECOG0.160.00–1.11–2.05–2.78-3.41
      BMI0.020.00–0.96–0.96–0.76–1.19
      Active Cancer0.040.000.760.580.510.23
      Metastatic0.030.000.920.640.630.24
      PFC45—Are you able to sit on and get up from the toilet?
      Items included in the final SF.
      KPS0.300.0014.0219.1023.3131.88
      ECOG0.130.00–3.08–4.05–4.77–6.34
      BMI0.010.000.170.33–0.28–0.71
      Active Cancer0.030.000.510.710.680.30
      Metastatic0.030.000.641.090.920.43
      PFA16—Are you able to dress yourself, including tying shoelaces and buttoning your clothes?
      Items included in the final SF.
      KPS0.310.006.0615.5019.5726.77
      ECOG0.120.00–1.54–3.03–3.76–4.97
      BMI0.010.000.410.47–0.07–0.24
      Active Cancer0.030.000.540.300.410.20
      Metastatic0.010.050.830.910.810.52
      PFA42—Are you able to carry a laundry basket up a flight of stairs?
      Items included in the final SF.
      KPS0.400.006.2810.7915.1120.46
      ECOG0.190.00–1.32–2.25–3.11–4.20
      BMI0.020.00–0.040.03–0.40–1.02
      Active Cancer0.030.000.530.440.480.31
      Metastatic0.020.001.000.700.580.39
      PFB12—Are you able to make a bed, including spreading and tucking in bed sheets?
      KPS0.360.007.7911.3415.3422.75
      ECOG0.160.00–1.87–2.24–3.16–4.56
      BMI0.010.00–0.100.20–0.47–0.55
      Active Cancer0.020.000.690.690.480.36
      Metastatic0.010.120.530.900.570.39
      PFA17—Are you able to reach into a high cupboard?
      KPS0.290.007.4910.8215.6019.67
      ECOG0.120.00–1.51–2.00–2.95–3.64
      BMI0.010.000.12–0.41–0.51–0.51
      Active Cancer0.010.070.840.820.620.57
      Metastatic0.000.421.030.920.650.81
      PFB18—Are you able to shave your face or apply make-up?
      KPS0.170.0011.9713.3816.3823.50
      ECOG0.060.00–2.05–2.14–2.79–3.87
      BMI0.010.01–1.04–0.92–1.22–1.29
      Active Cancer0.010.012.421.181.270.84
      Metastatic0.00.930.700.770.710.66
      PFC12—Does your health now limit you in doing two hours of physical labor?
      KPS0.340.008.6512.7316.5922.05
      ECOG0.150.00–1.44–2.20–3.08–3.91
      BMI0.010.00–0.28–0.49–0.53–0.91
      Active Cancer0.030.000.730.590.440.24
      Metastatic0.020.000.980.820.520.27
      PFA4—Does your health now limit you in doing heavy work around the house like scrubbing floors, or lifting or moving heavy furniture?
      Items included in the final SF.
      KPS0.330.007.6513.2716.5422.57
      ECOG0.150.00–1.35–2.53–3.14–4.28
      BMI0.010.000.02–0.40–0.80–0.57
      Active Cancer0.010.000.680.720.520.32
      Metastatic0.020.000.930.960.500.20
      FATIMP30—How often were you too tired to think clearly?
      KPS0.080.003.196.079.6412.12
      ECOG0.030.00–0.54–0.95–1.59–1.98
      BMI0.010.010.21–0.05–0.01–0.48
      Active Cancer0.000.991.030.950.900.97
      Metastatic0.000.930.820.861.010.93
      FATIMP51—How often did you have trouble finishing things because of your fatigue?
      Items included in the final SF.
      PS0.130.006.7310.1915.2317.90
      ECOG0.060.00–0.74–1.39–2.23–2.79
      BMI0.010.000.430.31–0.22–0.19
      Active Cancer0.030.001.071.700.860.56
      Metastatic0.020.000.670.910.500.35
      FATIMP18—How often did you have to limit your social activities because of your fatigue?
      Items included in the final SF.
      KPS0.170.007.3610.3715.5419.57
      ECOG0.090.00–1.37–1.82–2.74–3.60
      BMI0.000.20–0.15–0.22–0.37–0.53
      Active Cancer0.020.001.061.180.800.49
      Metastatic0.010.030.770.750.710.36
      FATEXP22—How often were you bothered by your fatigue?
      KPS0.140.005.558.9414.3716.39
      ECOG0.070.00–0.93–1.50–2.47–3.00
      BMI0.010.00–0.19–0.39–1.07–0.77
      Active Cancer0.020.000.981.160.610.42
      Metastatic0.010.020.610.840.540.33
      FATIMP9—How often did your fatigue make it difficult to plan activities ahead of time?
      Items included in the final SF.
      KPS0.170.005.558.6314.2517.24
      ECOG0.070.00–0.90–1.34–2.24–2.90
      BMI0.010.01–0.30–0.37–0.56–0.83
      Active Cancer0.030.000.850.990.570.34
      Metastatic0.020.000.800.840.540.35
      SRPPER11—I have trouble doing all of my regular leisure activities with others
      Items included in the final SF.
      Response option 1 was the comparator.
      KPS0.270.005.1210.9017.3920.65
      ECOG0.120.00–0.90–1.93–3.08–3.77
      BMI0.000.000.070.01–0.10–0.47
      Active Cancer0.030.000.630.640.350.25
      Metastatic0.020.000.860.840.550.28
      SRPPER16—I have to do my work for shorter periods of time than usual
      KPS0.270.008.1012.3518.6321.15
      ECOG0.130.00–1.46–2.34–3.59–4.10
      BMI0.010.000.320.19–0.18–0.61
      Active Cancer0.030.000.740.730.540.14
      Metastatic0.040.000.720.730.500.06
      SRPPER18—I have trouble doing all of the family activities that I want to do
      Items included in the final SF.
      KPS0.300.008.7414.0021.0023.50
      ECOG0.140.00–1.54–2.36–3.77–4.33
      BMI0.010.000.070.12–0.13–0.65
      Active Cancer0.040.000.430.600.310.17
      Metastatic0.040.000.330.530.340.10
      SRPPER23—I have trouble doing all of the work that is really important to me (including work at home)
      Items included in the final SF.
      KPS0.310.007.5513.5719.9522.98
      ECOG0.150.00–1.37–2.60–3.67–4.49
      BMI0.010.000.020.12–0.25–0.69
      Active Cancer0.030.000.670.770.370.20
      Metastatic0.030.000.620.770.450.11
      SRPPER46—I have trouble doing all of the activities with friends that I want to do
      KPS0.300.007.6012.5420.0722.68
      ECOG0.130.00–1.26–2.27–3.33–4.18
      BMI0.000.05–0.02–0.01–0.23–0.61
      Active Cancer0.020.000.510.580.350.21
      Metastatic0.040.000.480.660.430.07
      NOTE. Items are named by the PROMIS item IDs. Items were scored based on participant responses compared to KPS, ECOG, BMI, presence of active cancer, and presence of metastatic cancer. R2 was calculated with KPS because of it being a linear scale of 11 possible responses. Pseudo-R2 was calculated for the other responses due to… Coefficients related the relationship between function (higher numbers indicate higher patient-reported function) and the variables.
      Items included in the final SF.
      Response option 1 was the comparator.
      KPS and ECOG had strong associations between the 21 candidate items, after removing PFB18 (“Are you able to shave your face or apply make-up?”), as reflected in R2 values of 0.29 to 0.40 for KPS and pseudo R2 values of 0.12 to 0.21. Pseudo R2 values were estimated with ordinal logistic regression and were therefore lower than the R2 values estimated with linear regression. The Prob (F), the probability that the null hypothesis is true for the model, was <.001 for all models. With few exceptions, coefficient P values for the response options were also <.001.
      Although the regression model output was principally used to refine SF item selection, the high R2 and pseudo R2 values of the physical function and social role ability items in the KPS and ECOG models support the convergent validity of these domains in the final SF.
      BMI had the lowest association with participants’ responses. Although the Prob (F) was <.001 for all models, pseudo R2 values ranged from <.001 to .2, excepting PFA21 (“Are you able to go up and down stairs at a normal pace?”), which had a pseudo R2 value of 0.24.
      Presence of active or metastatic cancer had more variable P values and response coefficients were inconsistently associated with increasing trait severity for some items. Pseudo R2 values for these models were generally low (<.001-0.04).

      Final item selection

      In the event that multiple items offered comparable information and had similar regression model output, the group used a consensus process to determine the collection of high performing items that optimized subdomain coverage, provided clinically actionable information, and were not redundant. For example, both PFA21 (“Are you able to go up and down stairs at a normal pace?”) and PFB24 (“Are you able to run a short distance, such as to catch a bus?”) performed similarly on IRT and regression analyses, captured a high trait range, and associated strongly with performance status. PFB24 was ultimately selected because stair climbing is a key subdomain of physical function and has clear rehabilitation implications. Table 5 lists the final 12-item SF.
      Table 5Final, 12-item SF
      DomainItemResponse Categories
      Physical functionAre you able to go for a walk of at least 15 min?Without any difficulty (5), with a little difficulty (4), with some difficulty (3), with much difficulty (2), unable to do (1)
      Are you able to go up and down stairs at a normal pace?
      Are you able to sit on and get up from the toilet?
      Are you able to dress yourself, including tying shoelaces and buttoning your clothes?
      Are you able to carry a laundry basket up a flight of stairs?
      Does your health now limit you in doing heavy work around the house like scrubbing floors, or lifting or moving heavy furniture?Not at all (5), a little bit (4), somewhat (3), quite a lot (2), cannot do (1)
      FatigueIn the past 7 d, how often did you have trouble finishing things because of your fatigue?Never (1), rarely (2), sometimes (3), often (4), always (5)
      In the past 7 d, how often did your fatigue make it difficult to plan activities ahead of time?
      In the past 7 d, how often did you have to limit your social activities because of your fatigue?
      Social participationI have trouble doing all of my regular leisure activities with others.Never (5), rarely (4), sometimes (3), often (2), always (1)
      I have trouble doing all of the work that is really important to me (including work at home).
      I have trouble doing all of the family activities that I want to do.

      Scoring algorithm

      IRT modeled instruments can be scored using model estimates or through simplified score conversion tables. The former require that item responses be stored on a computing platform -programmed with the IRT model output. The CRMMC believed that this requirement would severely constrain use of the SF and, therefore, chose to work with the Outcomes and Measurement Science Hub within the Department of Medical Social Sciences, Northwestern Feinberg School of Medicine, to develop a score conversion chart using official PROMIS calibrations (appendix 2)

      Discussion

      To the best of our knowledge, the PROMIS Cancer Function Brief 3D Profile is the first specialized SF created from PROMIS item banks to meet the unique assessment and clinical decision-making needs of rehabilitation providers who treat cancer patients. We believe that the development of the PROMIS Cancer Function Brief 3D Profile is important, as it has high information density and requires minimal time for patients to complete.
      Specific and consistent subdomain coverage in PRO assessments is required to individualize management and ensure that providers receive reliable information regarding a patient’s function. Extant PROMIS CAT and SF tools either do not provide consistent (CAT) or specific (generic SFs) subdomain coverage, whereas the PROMIS Cancer Function Brief 3D Profile does. In addition, within a given domain such as fatigue, scores from this measure are comparable to those derived using PROMIS CATs and SFs, as all include IRT-calibrated items on a common unidimensional continuum. Because domain scores are comparable, data collected can track the trajectory of a patient’s longitudinal progress while providing information needed by diverse provider groups, primary and specialist, to inform clinical decision-making.
      The resulting 12-item SF met the goals the CRMMC set out to accomplish. It assesses key subdomains in trait ranges relevant to cancer survivors seen in outpatient clinics. Its developmental process was comprehensive, evidence-based, and used expert opinion to leverage vetted and highly applicable PROMIS items into a useful, short PRO tool. Furthermore, given its low respondent burden, many patients were enrolled and a high percentage of patients completed the items. Because administration and data capture did not significantly interfere with the workflow of study physicians, the PROMIS Cancer Function Brief 3D Profile is feasible to collect in busy outpatient clinics. Given the different performance sites and modes of collecting data for this study, the feasibility of this profile is likely generalizable to most practice settings.
      The benefits of this process are borne out in the study results, wherein the selected PROMIS items performed well vis-à-vis anchors (KPS and ECOG) and relevant clinical characteristics. In addition, it is the only SF that we know of to be developed and assessed in multiple cancer rehabilitation medicine programs, which minimizes sample bias and ensures utility in various outpatient practices. Although a large proportion of the study cohort were women with breast cancer, this weighting is consistent with supportive care clinical trials and representative of cancer rehabilitation care worldwide.

      Study limitations

      Limitations include that data were collected from clinics affiliated with large cancer centers. Therefore, the PROMIS Cancer Function Brief 3D Profile’s value in the community oncology setting is less clear. Furthermore, the tool was developed to err on the side of low respondent burden to increase the likelihood that the instrument would be administered, completed, and incorporated in clinical decision-making. Despite efforts to ensure comprehensive subdomain coverage, it is possible that functional information may be sacrificed for brevity. In addition, information was not collected regarding participants’ race, ethnicity, educational level, or socioeconomic status, so future PROMIS Cancer Function Brief 3D Profile assessments should evaluate its performance across minority and vulnerable subgroups. The fact that nonphysician rehabilitation service providers and patient representatives did not participate in the development of this tool is also a limitation. However, the instrument was constructed to support clinical assessment and decision-making by rehabilitation physicians. The CRMMC is hopeful that physical therapists, occupational therapists, and other providers will trial this tool to determine its validity in other settings.
      In the future, comparison of this SF with existing measures may further support its role in the evaluation of functional status in cancer patients. Future investigations should also explore its use toward triaging patients to cancer rehabilitation care.

      Conclusions

      The results of IRT analysis, multivariate regression analyses, and expert evaluation of the items that were field-tested produced a new 12-item assessment tool measuring physical function, fatigue, and social participation in patients with a history of cancer. The results support use of the PROMIS Cancer Function Brief 3D Profile for evaluating function in patients with a history of cancer.

      Suppliers

      • a.
        REDCap; Vanderbilt University.
      • b.
        Excel; Microsoft Corp.
      • c.
        STAT, v 15.0; STATACorp LLC.
      • d.
        flexMIRT, v 2.0; Vector Psychometric Group.

      Acknowledgments

      The authors thank Drs. Eric Wisotzky, MD, and Chris Custodio, MD, for their contributions to conceptualizing the assessment tool.

      Supplementary Data

      Appendix 1The Original 21 PROMIS Items Administered to Patients
      DomainItemResponse Categories
      Physical functionAre you able to go for a walk of at least 15 min? (PFA23)Without any difficulty (5), with a little difficulty (4), with some difficulty (3), with much difficulty (2), unable to do (1)
      Are you able to go up and down stairs at a normal pace? (PFA21)
      Are you able to run a short distance, such as to catch a bus? (PFB24)
      Are you able to sit on and get up from the toilet? (PFC45)
      Are you able to dress yourself, includ ing tying shoelaces and buttoning your clothes? (PFA16)
      Are you able to carry a laundry basket up a flight of stairs? (PFA42)
      Are you able to make a bed, including spreading and tucking in bed sheets? (PFB12)
      Are you able to reach into a high cupboard? (PFA17)
      Are you able to shave your face or apply make-up (PFB18)
      Does your health now limit you in doing two hours of physical labor? (PFC12)Not at all (5), a little bit (4), somewhat (3), quite a lot (2), cannot do (1)
      Does your health now limit you in doing heavy work around the house like scrubbing floors, or lifting or moving heavy furniture? (PFA4)
      FatigueIn the past 7 d, how often were you too tired to think clearly? (FATIMP30)Never (5), rarely (4), sometimes (3), often (2), always (1)
      In the past 7 d, how often did you have trouble finishing things because of your fatigue? (FATIMP51)
      In the past 7 d, how often did you have to limit your social activities because of your fatigue? (FATIMP18)
      In the past 7 d, how often were you bothered by your fatigue? (FATEXP22)
      In the past 7 d, how often did your fatigue make it difficult to plan activities ahead of time? (FATIMP9)
      Social participationI have trouble doing all of my regular leisure activities with others. (SRPPER11)Never (5), rarely (4), sometimes (3), often (2), always (1)
      I have to do my work for shorter periods of time than usual. (SRPPER16)
      I have trouble doing all of the family activities that I want to do. (SRPPER18)
      I have trouble doing all of the work that is really important to me (including work at home). (SRPPER23)
      I have trouble doing all of the activities with friends that I want to do. (SRPPER46)
      Appendix 2Scoring Algorithm for the PROMIS Cancer Function Brief 3D Profile
      Physical Function Short Form Conversion Table
      RawScaleSE
      617.93.8
      721.13.5
      823.23.3
      924.93
      1026.62.8
      1128.12.6
      1229.62.4
      1330.82.3
      1431.92.2
      15332.1
      1633.92.1
      1734.92.1
      1835.82.1
      1936.82.1
      2037.72.1
      2138.72.1
      2239.82.2
      2340.92.2
      24422.3
      2543.32.3
      2644.82.4
      2746.52.6
      2848.62.9
      2951.23.3
      3058.96.2
      Fatigue Short Form Conversion Table
      RawScaleSE
      338.55.8
      445.63.5
      548.93.2
      651.63.1
      754.13.1
      856.63.1
      9593
      1061.53.1
      1164.13.1
      1266.83.1
      1369.53.2
      1472.83.3
      1577.24
      Ability to Participate in Social Roles and Activities Short Form Conversion Table
      RawScaleSE
      328.14.2
      432.82.7
      535.32.5
      637.32.4
      739.42.5
      841.72.6
      944.22.6
      1046.82.6
      1149.32.5
      1251.72.5
      1354.12.6
      1457.12.9
      1563.45.2
      Abbreviation: SE, standard error on T-score metric.

      References

        • Miller K.D.
        • Siegel R.L.
        • Lin C.C.
        • et al.
        Cancer treatment and survivorship statistics, 2016.
        CA Cancer J Clin. 2016; 66: 271-289
        • Silver J.K.
        • Stout N.L.
        • Fu J.B.
        • Pratt-Chapman M.
        • Haylock P.J.
        • Sharma R.
        The state of cancer rehabilitation in the United States.
        J Cancer Rehabil. 2018; 1: 1-8
        • Wu H.S.
        • Harden J.K.
        Symptom burden and quality of life in survivorship: a review of the literature.
        Cancer Nurs. 2015; 38: E29-54
        • Holm L.V.
        • Hansen D.G.
        • Johansen C.
        • et al.
        Participation in cancer rehabilitation and unmet needs: a population-based cohort study.
        Support Care Cancer. 2012; 20: 2913-2924
        • Harrison J.D.
        • Young J.M.
        • Price M.A.
        • Butow P.N.
        • Solomon M.J.
        What are the unmet supportive care needs of people with cancer? A systematic review.
        Support Care Cancer. 2009; 17: 1117-1128
        • Alfano C.M.
        • Mayer D.K.
        • Bhatia S.
        • et al.
        Implementing personalized pathways for cancer follow-up care in the United States: proceedings from an American Cancer Society–American Society of Clinical Oncology summit.
        CA Cancer J Clin. 2019; 69: 234-247
        • Alfano C.M.
        • Mayer D.K.
        • Beckjord E.
        • et al.
        Mending disconnects in cancer care: setting an agenda for research, practice, and policy.
        JCO Clin Cancer Inform. 2020; 4: 539-546
        • Pergolotti M.
        • Alfano C.M.
        • Cernich A.N.
        • et al.
        A health services research agenda to fully integrate cancer rehabilitation into oncology care.
        Cancer. 2019; 125: 3908-3916
        • Smith S.R.
        • Vargo M.
        • Zucker D.S.
        • Henderson M.
        • Shahpar S.
        • Wisotzky E.M.
        • Custodio C.
        • Basford J.
        • Jay G.
        • Gerber L.
        • Cheville A.
        The Cancer Rehabilitation Medicine Metrics Consortium: A path to enhanced, multi-site outcome assessment to enhance care and demonstrate value.
        Front Oncol. 2021; 10: 625700
        • Neo J.
        • Fettes L.
        • Gao W.
        • Higginson I.J.
        • Maddocks M.
        Disability in activities of daily living among adults with cancer: a systematic review and meta-analysis.
        Cancer Treat Rev. 2017; 61: 94-106
        • Gerber L.H.
        Cancer-related fatigue: persistent, pervasive, and problematic.
        Phys Med Rehabil Clin. 2017; 28: 65-88
        • Jones J.M.
        • Olson K.
        • Catton P.
        • et al.
        Cancer-related fatigue and associated disability in post-treatment cancer survivors.
        J Cancer Surviv. 2016; 10: 51-61
        • Brinkman T.M.
        • Recklitis C.J.
        • Michel G.
        • Grootenhuis M.A.
        • Klosky J.L.
        Psychological symptoms, social outcomes, socioeconomic attainment, and health behaviors among survivors of childhood cancer: current state of the literature.
        J Clin Oncol. 2018; 36: 2190-2197
        • Jobst A.
        • Kirchberger I.
        • Cieza A.
        • Stucki G.
        • Stucki A.
        Content validity of the comprehensive ICF core set for chronic obstructive pulmonary diseases: an international Delphi survey.
        Open Respir Med J. 2013; 7: 33-45
        • Herrmann K.H.
        • Kirchberger I.
        • Stucki G.
        • Cieza A.
        The comprehensive ICF core sets for spinal cord injury from the perspective of physical therapists: a worldwide validation study using the Delphi technique.
        Spinal Cord. 2011; 49: 502-514
        • Glocker C.
        • Kirchberger I.
        • Gläßel A.
        • Fincziczki A.
        • Stucki G.
        • Cieza A.
        Content validity of the comprehensive international classification of functioning, disability and health (ICF) core set for low back pain from the perspective of physicians: a Delphi survey.
        Chronic Ill. 2013; 9: 57-72
        • Harris P.A.
        • Taylor R.
        • Thielke R.
        • Payne J.
        • Gonzalez N.
        • Conde J.G.
        Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42 (377-81)
        • Harris P.A.
        • Taylor R.
        • Minor B.L.
        • et al.
        The REDCap consortium: building an international community of software partners.
        J Biomed Inform. 2019; 95: 103208
        • Ainsworth B.E.
        • Haskell W.L.
        • Herrmann S.D.
        • et al.
        2011 compendium of physical activities: a second update of codes and MET values.
        Med Sci Sports Exerc. 2011; 43: 1575-1581
        • Holland J.C.
        • Bultz B.D.
        The NCCN guideline for distress management: a case for making distress the sixth vital sign.
        J Natl Compr Canc Netw. 2007; 5: 3-7
        • Schag C.C.
        • Heinrich R.L.
        • Ganz P.A.
        Karnofsky performance status revisited: reliability, validity, and guidelines.
        J Clin Oncol. 1984; 2: 187-193
        • Oken M.M.
        • Creech R.H.
        • Tormey D.C.
        • et al.
        Toxicity and response criteria of the Eastern Cooperative Oncology Group.
        Am J Clin Oncol. 1982; 5: 649-656
        • Lai J.S.
        • Cella D.
        • Choi S.
        • et al.
        How item banks and their application can influence measurement practice in rehabilitation medicine: a PROMIS fatigue item bank example.
        Arch Phys Med Rehabil. 2011; 92: S20-S27
        • Reeve B.B.
        • Hays R.D.
        • Bjorner J.B.
        • et al.
        Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS).
        Med Care. 2007; 45: S22-31