Advertisement
Departments Letter to the Editor| Volume 99, ISSUE 8, P1688-1689, August 2018

Download started.

Ok

Response to Letter “Prediction of Falls in Subjects Suffering From Parkinson Disease, Multiple Sclerosis, and Stroke: Methodologic Issues”

      We agree that the results of our study
      • Beghi E.
      • Gervasoni E.
      • Pupillo E.
      • et al.
      Prediction of falls in subjects suffering from Parkinson disease, multiple sclerosis, and stroke.
      are not appropriate for the prediction of an outcome, if prediction is intended as the process of applying a statistical model to the data, with the purpose of predicting new or future observations. When constructing a predictive model, the independent variables (predictors) should be selected based on the ability of the model to provide valid predictions (internal validity).
      • Altman D.G.
      • Royston P.
      What do we mean by validating a prognostic model?.
      • Steyerberg E.W.
      • Harrell Jr., F.E.
      • Borsboom G.J.
      • Eijkemans M.J.
      • Vergouwe Y.
      • Habbema J.D.
      Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.
      • Steyerberg E.W.
      • Vickers A.J.
      • Cook N.R.
      • et al.
      Assessing the performance of prediction models: a framework for traditional and novel measures.
      We understand that after a predictive model is developed, a fundamental issue is to verify the performance of the model when applied to a new population. The predictive performance should be first assessed through internal validation, using techniques such as split sample, cross-validation, or bootstrap resampling. External validation, applying the model to a new external dataset, is also required to determine generalizability. Performance should be quantified in terms of calibration (agreement between predicted probabilities and observed probabilities), discrimination (ability to distinguish high-risk subjects from low-risk subjects), and overall accuracy of predictions (distance between the predicted outcome and the actual outcome).
      • Altman D.G.
      • Royston P.
      What do we mean by validating a prognostic model?.
      • Steyerberg E.W.
      • Harrell Jr., F.E.
      • Borsboom G.J.
      • Eijkemans M.J.
      • Vergouwe Y.
      • Habbema J.D.
      Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.
      • Steyerberg E.W.
      • Vickers A.J.
      • Cook N.R.
      • et al.
      Assessing the performance of prediction models: a framework for traditional and novel measures.
      The development of a predictive model was not however the aim of our study. Our aims were to describe the risk of falls in 3 cohorts of patients with 3 different neurologic diseases (Parkinson disease, multiple sclerosis, and stroke), and to identify risk factors for falls in a set of demographic, clinical, and behavioral characteristics. The risk of falls in the 3 cohorts was described with Kaplan-Meier curves comparing the 3 diseases. We used univariable and multivariable Cox proportional hazard models to determine which independent variables (predictors) were associated with the outcome, and whether specific variables were associated with the outcome after allowance for other variables (possible confounders).
      • Altman D.G.
      • Royston P.
      What do we mean by validating a prognostic model?.
      We also recognize the importance of evaluating effect modification in addition to the correction of confounding. Double interaction terms were initially included in our models but were subsequently excluded as not statistically significant. In conclusion, we agree that association does not guarantee prediction, in the sense previously described. We used the term prediction intending the identification of predictors (independent variables) that were associated with the outcome and that could explain it. In light of the comments received in this letter, we agree that the choice of the term prediction was not appropriate in this context because it leads to the superimposition of 2 different research aims that require different statistical modeling approaches: the development of predictive models versus the use of regression models with the purpose of capturing the association between the dependent and independent variables.
      • Shmueli G.
      To explain or to predict?.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Beghi E.
        • Gervasoni E.
        • Pupillo E.
        • et al.
        Prediction of falls in subjects suffering from Parkinson disease, multiple sclerosis, and stroke.
        Arch Phys Med Rehabil. 2018; 99: 641-651
        • Altman D.G.
        • Royston P.
        What do we mean by validating a prognostic model?.
        Stat Med. 2000; 19: 453-473
        • Steyerberg E.W.
        • Harrell Jr., F.E.
        • Borsboom G.J.
        • Eijkemans M.J.
        • Vergouwe Y.
        • Habbema J.D.
        Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.
        J Clin Epidemiol. 2001; 54: 774-781
        • Steyerberg E.W.
        • Vickers A.J.
        • Cook N.R.
        • et al.
        Assessing the performance of prediction models: a framework for traditional and novel measures.
        Epidemiology. 2010; 21: 128-138
        • Shmueli G.
        To explain or to predict?.
        Stat Sci. 2010; 25: 289-310

      Linked Article

      • Prediction of Falls in Subjects Suffering From Parkinson Disease, Multiple Sclerosis, and Stroke: Methodologic Issues
        Archives of Physical Medicine and RehabilitationVol. 99Issue 8
        • Preview
          I was interested to read the article by Beghi et al1 published in the April 2018 issue of Archives. The authors aimed to compare the risk of falls and fall predictors in patients with Parkinson disease (PD), multiple sclerosis (MS), and stroke. They included a total of 299 patients as follows: PD (n=94), MS (n=111), and stroke (n=94). They applied functional scales to investigate balance, disability, daily performance, self-confidence with balance, and social integration. Patients were followed for 6 months.
        • Full-Text
        • PDF