Using Wearable Sensors and Machine Learning Models to Separate Functional Upper Extremity Use From Walking-Associated Arm Movements

Published:October 02, 2015DOI:



      To improve measurement of upper extremity (UE) use in the community by evaluating the feasibility of using body-worn sensor data and machine learning models to distinguish productive prehensile and bimanual UE activity use from extraneous movements associated with walking.


      Comparison of machine learning classification models with criterion standard of manually scored videos of performance in UE prosthesis users.


      Rehabilitation hospital training apartment.


      Convenience sample of UE prosthesis users (n=5) and controls (n=13) similar in age and hand dominance (N=18).


      Participants were filmed executing a series of functional activities; a trained observer annotated each frame to indicate either UE movement directed at functional activity or walking. Synchronized data from an inertial sensor attached to the dominant wrist were similarly classified as indicating either a functional use or walking. These data were used to train 3 classification models to predict the functional versus walking state given the associated sensor information. Models were trained over 4 trials: on UE amputees and controls and both within subject and across subject. Model performance was also examined with and without preprocessing (centering) in the across-subject trials.

      Main Outcome Measure

      Percent correct classification.


      With the exception of the amputee/across-subject trial, at least 1 model classified >95% of test data correctly for all trial types. The top performer in the amputee/across-subject trial classified 85% of test examples correctly.


      We have demonstrated that computationally lightweight classification models can use inertial data collected from wrist-worn sensors to reliably distinguish prosthetic UE movements during functional use from walking-associated movement. This approach has promise in objectively measuring real-world UE use of prosthetic limbs and may be helpful in clinical trials and in measuring response to treatment of other UE pathologies.


      List of abbreviations:

      ADL (activities of daily living), IMU (inertial measurement unit), UE (upper extremity)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Dobkin B.H.
        Wearable motion sensors to continuously measure real-world physical activities.
        Curr Opin Neurol. 2013; 26: 602-608
        • Lang C.E.
        • Wagner J.M.
        • Dromerick A.W.
        • Edwards D.F.
        Measurement of upper-extremity function early after stroke: properties of the action research arm test.
        Arch Phys Med Rehabil. 2006; 87: 1605-1610
        • Desrosiers J.
        • Bravo G.
        • Hebert R.
        • Dutil E.
        • Mercier L.
        Validation of the Box and Block Test as a measure of dexterity of elderly people: reliability, validity, and norms studies.
        Arch Phys Med Rehabil. 1994; 75: 751-755
        • Lang C.E.
        • Wagner J.M.
        • Bastian A.J.
        • et al.
        Deficits in grasp versus reach during acute hemiparesis.
        Exp Brain Res. 2005; 166: 126-136
        • Katz S.
        Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living.
        J Am Geriatr Soc. 1983; 31: 721-727
        • Dromerick A.W.
        • Edwards D.F.
        • Diringer M.N.
        Sensitivity to changes in disability after stroke: a comparison of four scales used in clinical trials.
        J Rehabil Res Dev. 2003; 40: 1-8
        • Dromerick A.W.
        • Lang C.E.
        • Birkenmeier R.
        • Hahn M.G.
        • Sahrmann S.A.
        • Edwards D.F.
        Relationships between upper-limb functional limitation and self-reported disability 3 months after stroke.
        J Rehabil Res Dev. 2006; 43: 401-408
        • Baum C.M.
        • Edwards D.
        ACS: Activity Card Sort.
        Washington School of Medicine at Washington University Medical Center, St. Louis2001
        • Mahoney F.I.
        • Barthel D.W.
        Functional evaluation: the Barthel Index.
        Md State Med J. 1955; 14: 61-65
        • Granger C.V.
        • Hamilton B.B.
        • Linacre J.M.
        • Heinemann A.W.
        • Wright B.D.
        Performance profiles of the Functional Independence Measure.
        Am J Phys Med Rehabil. 1993; 72: 84-89
        • Desmond D.
        • MacLachlan M.
        Factor Structure of the Trinity Amputation and Prosthesis Experience Scales (TAPES) with individuals with acquired upper limb amputations.
        Am J Phys Med Rehabil. 2005; 84: 506-513
        • Uswatte G.
        • Taub E.
        • Morris D.
        • Light K.
        • Thompson P.A.
        The Motor Activity Log-28: assessing daily use of the hemiparetic arm after stroke.
        Neurology. 2006; 67: 1189-1194
        • Duncan P.W.
        • Wallace D.
        • Lai S.M.
        • Johnson D.
        • Embretson S.
        • Laster L.J.
        The Stroke Impact Scale Version 2.0. Evaluation of reliability, validity, and sensitivity to change.
        Stroke. 1999; 30: 2131-2140
        • van der Pas S.C.
        • Verbunt J.A.
        • Breukelaar D.E.
        • van Woerden R.
        • Seelen H.A.
        Assessment of arm activity using triaxial accelerometry in patients with a stroke.
        Arch Phys Med Rehabil. 2011; 92: 1437-1442
        • Wolfe F.
        • Michaud K.
        The Hawthorne effect, sponsored trials, and the overestimation of treatment effectiveness.
        J Rheumatol. 2010; 37: 2216-2220
        • Godfrey A.
        • Conway R.
        • Meagher D.
        • OLaighin G.
        Direct measurement of human movement by accelerometry.
        Med Eng Phys. 2008; 30: 1364-1386
        • Bonato P.
        Advances in wearable technology and its medical applications.
        Conf Proc IEEE Eng Med Biol Soc. 2010; 2010: 2021-2024
        • Bourke A.K.
        • O'Brien J.V.
        • Lyons G.M.
        Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm.
        Gait Posture. 2007; 26: 194-199
        • Uswatte G.
        • Foo W.L.
        • Olmstead H.
        • Lopez K.
        • Holand A.
        • Simms L.B.
        Ambulatory monitoring of arm movement using accelerometry: an objective measure of upper-extremity rehabilitation in persons with chronic stroke.
        Arch Phys Med Rehabil. 2005; 86: 1498-1501
        • Lang C.E.
        • Wagner J.M.
        • Edwards D.F.
        • Dromerick A.W.
        Upper extremity use in people with hemiparesis in the first few weeks after stroke.
        J Neurol Phys Ther. 2007; 31: 56-63
        • Uswatte G.
        • Hobbs Qadri L.
        A behavioral observation system for quantifying arm activity in daily life after stroke.
        Rehabil Psychol. 2009; 54: 398-403
      1. Lester J, Choudhury T, Kern N, Borriello G, Hannaford B. A hybrid discriminative/generative approach for modeling human activities. In: Proceedings of the 19th International Joint Conferences on Artificial Intelligence; 2005 Jul 30-Aug 5; Edinburgh (Scotland). p 766-72.

        • Bao L.
        • Intille S.S.
        Activity recognition from user-annotated acceleration data. Pervasive computing.
        Springer, Heidelberg2004
        • Maurer U.
        • Smailagic A.
        • Siewiorek D.P.
        • Deisher M.
        Activity recognition and monitoring using multiple sensors on different body positions.
        in: International Workshop on Wearable and Implantable Body Sensor Networks, 2006. BSN 2006. 2006 Apr 3-5. IEEE, New York2006: 112-116
        • Hong Y.J.
        • Kim I.J.
        • Ahn S.C.
        • Kim H.G.
        Activity recognition using wearable sensors for elder care.
        in: Second International Conference on Future Generation Communication and Networking, 2008. FGCN'08. 2008 Dec 13-15. IEEE, New York2008: 112-116
        • Lee S.W.
        • Mase K.
        Activity and location recognition using wearable sensors.
        IEEE Pervasive Comput. 2002; 1: 24-32
        • Giuffrida J.P.
        • Lerner A.
        • Steiner R.
        • Daly J.
        Upper-extremity stroke therapy task discrimination using motion sensors and electromyography.
        IEEE Trans Neural Syst Rehabil Eng. 2008; 16: 82-90
        • Stikic M.
        • Huynh T.
        • Van Laerhoven K.
        • Schiele B.
        ADL recognition based on the combination of RFID and accelerometer sensing.
        in: Proceedings of the 2nd International Conference on Pervasive Computing Technologies for Healthcare, 2008. 2008 Jan 30-Feb 1. IEEE, New York2008: 258-263
        • Tapia E.M.
        • Intille S.S.
        • Haskell W.
        • et al.
        Real-time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor.
        in: 11th IEEE International Symposium on Wearable Computers; 2007 Oct 11-13. IEEE, New York2007: 37-40
        • Zhu C.
        • Sheng W.
        Human daily activity recognition in robot-assisted living using multi-sensor fusion.
        in: IEEE International Conference on Robotics and Automation, 2009. ICRA'09. 2009 May 12-17. IEEE, New York2009: 2154-2159
        • Green L.B.
        Assessment of habitual physical activity and paretic arm mobility among stroke survivors by accelerometry.
        Top Stroke Rehabil. 2007; 14: 9-21
        • Huỳnh T.
        • Blanke U.
        • Schiele B.
        Scalable recognition of daily activities with wearable sensors. Location-and context-awareness.
        Springer, Heidelberg2007
        • Wang S.
        • Yang J.
        • Chen N.
        • Chen X.
        • Zhang Q.
        Human activity recognition with user-free accelerometers in the sensor networks.
        in: 2005 International Conference on Neural Networks and Brain, 2005. ICNN&B'05. 2005 Oct 13-15. IEEE, New York2005: 1212-1217
        • Pirttikangas S.
        • Fujinami K.
        • Nakajima T.
        Feature selection and activity recognition from wearable sensors. Ubiquitous computing systems.
        Springer, Heidelberg2006
        • Uswatte G.
        • Miltner W.H.
        • Foo B.
        • Varma M.
        • Moran S.
        • Taub E.
        Objective measurement of functional upper-extremity movement using accelerometer recordings transformed with a threshold filter.
        Stroke. 2000; 31: 662-667
        • Ermes M.
        • Parkka J.
        • Mantyjarvi J.
        • Korhonen I.
        Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions.
        IEEE Trans Inf Technol Biomed. 2008; 12: 20-26
        • Bailey R.R.
        • Klaesner J.W.
        • Lang C.E.
        An accelerometry-based methodology for assessment of real-world bilateral upper extremity activity.
        PLoS One. 2014; 9: e103135
        • Patel S.
        • Hughes R.
        • Hester T.
        • et al.
        A novel approach to monitor rehabilitation outcomes in stroke survivors using wearable technology.
        Proceedings of the IEEE. 2010; 98: 450-461
        • Wolf S.L.
        • McJunkin J.P.
        • Swanson M.L.
        • Weiss P.S.
        Pilot normative database for the wolf motor function test.
        Arch Phys Med Rehabil. 2006; 87: 443-445
        • Siirtola P.
        • Laurinen P.
        • Haapalainen E.
        • Roning J.
        • Kinnunen H.
        Clustering-based activity classification with a wrist-worn accelerometer using basic features.
        in: IEEE Symposium on Computational Intelligence and Data Mining, 2009. CIDM'09. 2009 Mar 30-Apr 2. IEEE, New York2009: 95-100
        • Parkka J.
        • Ermes M.
        • Korpipaa P.
        • Mantyjarvi J.
        • Peltola J.
        • Korhonen I.
        Activity classification using realistic data from wearable sensors.
        IEEE Trans Inf Technol Biomed. 2006; 10: 119-128
        • Hall M.
        • Frank E.
        • Holmes G.
        • Pfahringer B.
        • Reutemann P.
        • Witten I.H.
        The WEKA data mining software: an update.
        ACM SIGKDD Explorations Newsletter. 2009; 11: 10-18
        • Le Cessie S.
        • Van Houwelingen J.
        Ridge estimators in logistic regression.
        Appl Stat. 1992; : 191-201
        • Quinlan J.R.
        C4. 5: programs for machine learning.
        Morgan Kaufmann, San Francisco1993
        • Wiktorin C.
        • Mortimer M.
        • Ekenvall L.
        • Kilbom Å.
        • Hjelm E.W.
        HARBO, a simple computer-aided observation method for recording work postures.
        Scand J Work Environ Health. 1995; 21: 440-449