If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding author Hyung-Soon Park, PhD, Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Mechanical Engineering Building (N7-3), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Korea.
Artificial intelligence was trained to mimic spasticity assessment by humans.
•
The artificial intelligence achieved satisfactory agreement with multiple raters.
•
The contribution of 9 characteristics of spasticity to modified Ashworth scale (MAS) rating was quantified.
•
Each characteristic contributed differently to the decision of different MAS grades.
•
Application of artificial intelligence for clinical assessment was discussed.
Abstract
Objective
To propose an artificial intelligence (AI)-based decision-making rule in modified Ashworth scale (MAS) that draws maximum agreement from multiple human raters and to analyze how various biomechanical parameters affect scores in MAS.
Design
Prospective observational study.
Setting
Two university hospitals.
Participants
Hemiplegic adults with elbow flexor spasticity due to acquired brain injury (N=34).
Intervention
Not applicable.
Main Outcome Measures
Twenty-eight rehabilitation doctors and occupational therapists examined MAS of elbow flexors in 34 subjects with hemiplegia due to acquired brain injury while the MAS score and biomechanical data (ie, joint motion and resistance) were collected. Nine biomechanical parameters that quantify spastic response described by the joint motion and resistance were calculated. An AI algorithm (or artificial neural network) was trained to predict the MAS score from the parameters. Afterwards, the contribution of each parameter for determining MAS scores was analyzed.
Results
The trained AI agreed with the human raters for the majority (82.2%, Cohen’s kappa=0.743) of data. The MAS scores chosen by the AI and human raters showed a strong correlation (correlation coefficient=0.825). Each biomechanical parameter contributed differently to the different MAS scores. Overall, angle of catch, maximum stretching speed, and maximum resistance were the most relevant parameters that affected the AI decision.
Conclusions
AI can successfully learn clinical assessment of spasticity with good agreement with multiple human raters. In addition, we could analyze which factors of spastic response are considered important by the human raters in assessing spasticity by observing how AI learns the expert decision. It should be noted that few data were collected for MAS3; the results and analysis related to MAS3 therefore have limited supporting evidence.
Spasticity is a common sequela after acquired brain injuries, which influences activities of daily living. It is defined as involuntary muscle activation due to abnormal sensory-motor control resulted from upper motor neuron lesion.
One particular characteristic of spastic muscle is sudden resistance (named catch) and/or increase of tone subjected to passive stretching. Reliable assessment of spasticity is essential to determine the proper treatment and know its efficacy. However, modified Ashworth scale (MAS)
There are 2 approaches to enhance the poor interrater reliability. First, quantitative and objective measurement can replace or augment the current subjective instrument. Second, without changing the simplest protocol, interrater reliability can still be enhanced by standardized training based on quantitative and statistical analysis. This paper presents artificial intelligence (AI)-based analysis for both approaches.
Quantitative measures of spasticity have been developed based on different approaches. The first approach quantified onset, latency, and magnitude of neural reflex using electromyography.
Another approach examined joint properties such as stiffness under controlled kinematic conditions including isokinetic stretching or random perturbation.
Although various quantitative measures have been proposed, clinical practices are still based on MAS presumably due to an unclear relationship between the suggested measures and MAS ratings. For a smooth and systematic transition from the old to the new, a user-friendly introduction of new quantitative measures should provide quantitative and statistical analysis explaining how previously used MAS rating system can be interpreted in the suggested system.
Beyond quantification of spasticity, there were few attempts to mimic clinical assessment based on statistical classification methods.
They performed simple classification (eg, severe vs nonsevere spasticity or subjects with spasticity vs healthy controls) based on quantitative parameters in literature. Compared to these approaches, an artificial neural network, an AI algorithm, can learn more complex tasks such as prediction of MAS score. It also allows quantitative analysis of how the AI algorithm learns the human decision-making process. This would be useful to design standardized training if AI learns what majority experts do.
This paper proposes an AI-based approach to understand how a majority of human experts decide MAS scores in common by evaluating contributions of various characteristic features quantified by biomechanical parameters. The AI-based analysis can either be used to describe standardized feeling of each MAS rating for consistent training of raters that would eventually improve interrater reliability of MAS itself or be used for systematic transition from the MAS to new quantitative instruments. Specifically, we collected joint motion, resistance, and MAS scores simultaneously from 848 trials conducted by multiple clinicians. Nine biomechanical parameters quantified the joint motion and resistance. An AI algorithm was then trained to predict the MAS score from the parameters. We quantitatively analyzed which parameters are considered important.
Methods
Participants
Hemiplegic adults with elbow flexor spasticity due to acquired brain injuries were recruited from 2 hospitals (Presbyterian Medical Center, Jeonju, Korea, and Chung-Ang University Hospital, Seoul, Korea). Inclusion criteria were as follows: (1) acquired brain injuries confirmed by computer tomography or magnetic resonance imaging; (2) more than 2 weeks after onset; (3) cognitive ability confirmed by Mini-Mental State Examination (score>18, Chung-Ang University Hospital) or by testing a 3-step command (Presbyterian Medical Center). Exclusion criteria were as follows: (1) no passive movement (ie, MAS4); (2) other neurological or orthopedic conditions that could affect testing the limb. The experimental protocol was approved by institutional review boards of the hospitals and every subject gave written consent. A total of 34 subjects (table 1) and a total of 28 rehabilitation doctors and occupational therapists (with an average career of 7.6y) participated.
was used to collect joint angle and resistance (fig 1). The forearm brace including the hand support was carefully designed to minimize slippage. At the same time, the device was designed to be light to minimize distortion of force perception. After wearing the device and fastening by straps, the joint was maximally flexed vertically to the initial posture (shoulder flexion of 40 degrees, abduction of 20 degrees and sitting on the chair), selected by referring to previous studies.
While extending the joint maximally, the raters held the handle of the device so that the resistive force (felt by the raters) was measured by a force sensora beneath the handle. The force reading was converted to torque about elbow joint afterwards. The joint angle was measured by an angle sensor.b The raters assessed spasticity for 5 trials and recorded MAS scores for each trial. Between trials, rest of 2 to 5 seconds was given randomly to prevent the subject from predicting the stretching. At least 3 raters (4.9 raters on average) assessed a subject. A rest of 5 minutes was given between raters. Prior to participation, raters were trained and became accustomed to the experimental protocol.
Fig 1Spasticity assessment using the manual spasticity evaluator. The rater extends the subject’s elbow joint while holding the handle of the device. The angle sensor aligned to the subject’s elbow joint measures angle of the joint and the force sensor beneath the handle (area surrounded by a dotted line) measures the force applied to the subject. The device is fastened to the subject’s upper arm, forearm, and hand using fabric straps.
Joint motion and resistance were quantified by 9 biomechanical parameters using custom MATLABc codes. Eight of them were adopted from the literature. Three joint motion parameters are passive range of motion (PROM), the difference between the minimum and maximum extension angles
; and angle of catch based on the maximum deceleration, the angular displacement from the initial angle to the joint angle when the largest deceleration occurs, which was normalized by PROM
; the peak resistance torque due to catch divided by the maximum stretching speed (fig 2D); and the slope of linear regression of the resistance with respect to the joint angle
(fig 2A). Another 2 parameters related to mechanical power (ie, multiplication of resistance and angular velocity) are angle of catch based on the local minimum of the mechanical power,
Fig 2Nine biomechanical parameters to quantify the spastic response. (A) PROM and resistance related parameters (slope of linear regression of the resistance with respect to the joint angle, magnitude of catch, resistance after catch, and maximum resistance during the passive stretching); (B) speed related parameters, maximum stretching speed; (C) acceleration related parameters, angle of catch based on the maximum deceleration; (D) mechanical power related parameters, angle of catch based on the local minimum of the mechanical power and local minimum of mechanical power; (E) detailed explanation on calculation of resistance after catch and magnitude of catch.
For example, it is described as catch and release (MAS1), catch followed by minimal resistance (MAS1+), or increase of muscle tone (MAS2, 3). To express the different shapes of resistance after the catch as a numeric value, we considered the amount of torque variation as well as the amount of angular variation since the raters would recognize the torque variation more dominant if the accompanied angular variation is larger. Figure 2E describes how we calculated resistance after catch. All parameters were transformed to have 0 means and unit variances to prevent the AI algorithm from biasing toward parameters with larger scales.
) predicted MAS score from the 9 biomechanical parameters. It resembles a multilayer of biological neurons. We developed a perceptron which has 9 neurons in each of the first 2 layers and 5 output neurons representing MAS0-3 in the last layer. Each neuron linearly combined outputs of neurons in the previous layer (or the biomechanical parameters) with its own weights and bias. Then the result of linear combination was saturated. Hyperbolic tangent functions were used for the first and second layers, while a log softmax function was used for the last layer in order that the output neurons had values ranging from 0 to 1, the sum of which was 1. Throughout the training, weights and biases were tuned to make the output neuron of the desired MAS score produce the maximum output value since the MAS score corresponding to the maximum output will be selected as the predicted score for the given sample. The weights and biases were modified after the prediction of each sample based on the gradient descent method (learning rate of 10-5). The training was repeated for 30,000 epochs, enough for the training result to converge. Note that all samples were used for the training. The perceptron we used was the simplest to achieve agreement with the human raters for more than 80% of trials, which is equal to or higher than those of previous studies.
In addition to the percentage agreement, Cohen’s kappa and Spearman rank correlation was used to evaluate the performance of the AI algorithm. Since single training might not represent averaged performance, 30 copies of the AI algorithm were trained separately and their results were averaged.
The contribution of the 9 parameters on the MAS scoring was reversely traced using layerwise relevance propagation.
Relevance of each neuron and the bias in 1 layer was calculated by distributing the relevance value of each neuron in the next layer according to their contribution to the linear combination for the neuron of the next layer (ie, multiplication of the output of each neuron and the corresponding weight, divided by the linearly combined value). To evaluate contribution on the predicted MAS score only, the value of the output neuron corresponding to the predicted score was distributed to the previous layers. Positive relevance means that the parameter strengthens the prediction whereas negative relevance weakens. The relevance value of each parameter was averaged over trials classified as the same score. One sample t test was used to test whether each parameter has nonzero average relevance for each score. We defined the overall relevance of each parameter as the sum of the average relevance values over the 5 MAS scores. Based on the overall relevance, the most and least relevant parameters were identified. The AI algorithm was developed and analyzed using custom MATLABc codes.
Results
Data collection
A total of 848 trials were collected since 1 rater conducted 3 trials at his or her first participation. From MAS0 to 3, 117, 324, 281, 96, and 30 trials were collected respectively. For most MAS3 trials, a catch was not observed due to slow stretching. For those trials, angle of catch based on the maximum deceleration and angle of catch based on the local minimum of the mechanical power were set to negative extreme values of -10 since they decrease as the MAS score increases. Similarly, magnitude of catch, resistance after catch, and local minimum of mechanical power were set to positive extreme values, +10. For these parameters, the average relevance for MAS3 was excluded from the overall relevance calculation since their relevance might be exaggerated due to the artificial extreme values. For MAS0, trials where the catch was observed were included in the further analysis since weak spasticity can sometimes be rated as MAS0. Trials with data error were excluded. The error includes malfunction of the sensors, failure to hold the handle while stretching and unsure MAS rating. A total of 648 trials (MAS0: 71, 1: 256, 1+: 224, 2: 78, 3: 19) were used to train the AI. Note that the following results about MAS3 were driven from a small number of samples.
Artificial neural network-based quantitative analysis of MAS
The AI algorithm successfully learned MAS rating. Percentage agreement of 82.2% and Cohen’s kappa of 0.743 (95% CI: [0.735, 0.751]) were achieved on average. Percentage agreement values for MAS0-3 were 76.7%, 80.4%, 85.0%, 84.9%, and 88.4%, respectively. It was strongly correlated with the human raters (correlation coefficient=0.825). Each parameter and its relevance were averaged over trials classified as the same MAS score (fig 3, 4). Despite the parameters with clear positive or negative relevance, some parameters showed insignificant relevance which had positive or negative relevance depending on trials. Angle of catch based on the maximum deceleration, resistance after catch, maximum resistance during the passive stretching, and slope of linear regression of the resistance with respect to the joint angle were relevant to the decision of MAS0. In the case of MAS2, angle of catch based on the maximum deceleration, resistance after catch, maximum resistance during the passive stretching, and maximum stretching speed were relevant. Magnitude of catch, resistance after catch, PROM, maximum stretching speed, and slope of linear regression of the resistance with respect to the joint angle were relevant to MAS3. Most parameters had negative relevance to MAS1 and 1+ while the biases for the linear combinations at the 3 layers had positive relevance.
Fig 3Distribution of biomechanical parameters of trials classified as each MAS score. Normalized values are shown. Zero level implies the mean value of each parameter. Dots indicate average values of each MAS score. Vertical lines show 1 SD. (A) angle of catch based on the maximum deceleration; (B) magnitude of catch; (C) resistance after catch; (D) PROM; (E) maximum resistance during the passive stretching; (F) maximum stretching speed; (G) angle of catch based on the local minimum of the mechanical power; (H) local minimum of mechanical power; (I) slope of linear regression of the resistance with respect to the joint angle. ∗For some MAS3 trials that catch was not clearly observed, angle of catch based on the maximum deceleration, angle of catch based on the local minimum of the mechanical power, magnitude of catch, resistance after catch, and local minimum of mechanical power were set to be extreme values (eg, -10 or 10).
Fig 4The average relevance of biomechanical parameters and biases to the decision of each MAS score. Red and blue colors indicate positive average relevance (relevant for the prediction) and negative average relevance (against the prediction), respectively. Overall relevance is defined as the sum of the average relevance values over the MAS scores. Relevance values of angle of catch based on the maximum deceleration, magnitude of catch, resistance after catch, angle of catch based on the local minimum of the mechanical power, and local minimum of mechanical power for MAS3 (colored in gray) were excluded from the calculation of overall relevance. ∗The average relevance value is not significantly different from 0 according to 1 sample t test (statistical significance: 5%).
Angle of catch based on the maximum deceleration, maximum resistance during the passive stretching, and maximum stretching speed were identified as the most relevant 3 parameters based on the overall relevance. With these parameters, the AI algorithm could achieve an agreement of 71.8%. On the other hand, the least relevant parameters (magnitude of catch, angle of catch based on the local minimum of the mechanical power, and local minimum of mechanical power) could achieve an agreement of 64.6%.
Discussion
Relevance of biomechanical parameters
Angle of catch based on the maximum deceleration was the most relevant to MAS0 and 2 (large and small angles of catch respectively, the normalized values far from 0 as shown in fig 3A). As angle of catch has been considered one of the primary factors in the spasticity assessment,
our result supports that angle of catch based on the maximum deceleration is specifically effective for determining MAS0 and 2. Low slope of linear regression of the resistance with respect to the joint angle (small increase of resistance throughout range of motion) and low maximum resistance during the passive stretching (smaller maximum resistance) were relevant features of MAS0, while high maximum resistance during the passive stretching, low maximum stretching speed (slow stretching), and high resistance after catch (increase of resistance after catch) were relevant to MAS2.
MAS1 and 1+ had only a few parameters which showed positive but not strong relevance. For example, in the case of angle of catch based on the maximum deceleration, a trial tended to be classified as MAS0 or 2 as the parameter deviated further from its mean. Thus, they can be considered as intermediate levels between MAS0 and 2. While most parameters worked similarly as angle of catch based on the maximum deceleration did, few parameters such as maximum resistance during the passive stretching for MAS1, and slope of linear regression of the resistance with respect to the joint angle for 1+ weakly contributed. Regarding the ambiguity between MAS1 and 1+,
understanding them requires a careful analysis, which can be a topic for the next study.
In the case of MAS3, PROM was prominently reduced. A catch was not observed clearly in most trials. Hence, we inevitably set the catch-related parameters (ie, angle of catch based on the maximum deceleration, magnitude of catch, resistance after catch, angle of catch based on the local minimum of the mechanical power, and local minimum of mechanical power) as the extreme values, which were clearly separable from the other trials. We found that magnitude of catch and resistance after catch, which had extreme values implying no catch, and slow maximum stretching speed were considered important.
Interestingly, maximum stretching speed was relevant to severe spasticity of MAS2 and 3. Raters stretched the severely impaired joints slowly since they might be stiffer. Considering the velocity dependency of spasticity, it is important to cope with the varying stretching speed. In this study, the speed was considered in decision-making rather than regulating the speed based on visual or auditory feedback.
When fewer parameters were used to train the AI, classification performance was degraded; however, better performance was achieved when the most relevant parameters were used.
Discussions on the AI-based approach
The AI-based approach enables a detailed quantitative analysis of human decision-making or even the decision-making by itself. With the conventional methods such as correlation or regression analyses, prediction of the clinical scores is still ambiguous. In this study, we found how much human raters rely on the specific parameter as well as ranges of the parameters corresponding to each MAS score (fig 3). In addition, if automated prediction like what IBM Watson does is required, different AI algorithm such as convolutional neural network specialized for multidimensional data can directly predict the MAS score from the response of spastic joint.
Regarding the clinical application of the AI algorithm, there are several points to discuss. First, the size and quality of dataset determine the prediction performance. More data guarantee better performance in general. The quality is improved as the data are collected from diverse cases and as answers given to the dataset (eg, human-rated MAS score) are consistent. For the diversity, we collected data from multiple subjects and raters. For the consistency, if inevitable, screening abnormal data may be helpful as we excluded trials with data error.
Second, using more neurons and layers can improve the learning capacity of the AI algorithm; however, the chance of learning errors in data may also increase. We had to address the dilemma of how to set the number of neurons and layers of the MLP. We collected MAS score for every trial to cope with intrasubject variation. However, if raters made inconsistent ratings, the more complex AI would learn the specific cases. We therefore used the simplest MLP in this study.
Lastly, the training result varies since the training process is stochastic. The AI algorithm is randomly initialized and guided by samples presented in random order. Thus, it can be trained differently even with the same architecture and dataset. To handle this variability, averaging the results of separate trainings is recommended. In this study, we averaged 30 separate trainings.
Study limitations
First, the measurement device might have caused bias of the MAS ratings. The measurement device was designed to minimize slippage between the device and the subjects as well as inertia of the device. In addition, we surveyed on the questionnaire: “Would you have rated the same MAS scores with the bare hand contact?” With 5 being exactly the same and 0 being completely different, we obtained 4.67 on average, which implies that the raters did not think that the device affected their rating. Despite the potential bias, the device can detect intrasubject variation which is another source of poor reliability. The effect of intrasubject variation can be minimized by averaging multiple measurements.
Second, analysis of MAS3 was based on relatively small samples since it was hard to find subjects with severe spasticity which is possibly due to the recent advances in the management of spasticity. Within the small number of samples, the artificial neural network learned the slow stretching speed as a clear feature of MAS3. This will still be the same with more MAS3 samples; however, the AI algorithm would find the more accurate boundary of parameters.
Last, we did not focus on the robustness of the AI rule since this study aimed at understanding how AI learns the majority of human experts. Therefore, validation using separate dataset was not conducted.
Conclusion
In this study, an AI-based approach was used to infer the MAS scoring rule from biomechanical data collected from multiple raters as well as multiple subjects. The AI achieved satisfactory performance of agreement (82.2%, kappa=0.743) and correlation (correlation coefficient of 0.825) with the human raters. Among the 9 parameters, angle of catch, maximum resistance, and maximum stretching speed were the most relevant to MAS. MAS0, 2, and 3 have distinct apparent patterns of the biomechanical parameters, whereas MAS1 and 1+ do not. These findings can be used for designing more standardized training of MAS for improving MAS instrument itself or for providing a systematic transition to other quantitative instruments. Furthermore, the AI-based relevance test can be applied to analyze the subjective nature of other clinical instruments.
Suppliers
a.
LRF325; Futek Advanced Sensor Technology Inc.
b.
HEDM-5500; Avago Technologies US Inc.
c.
MATLAB 2018a; Mathworks Inc.
References
Burridge J.H.
Wood D.E.
Hermens H.J.
et al.
Theoretical and methodological considerations in the measurement of spasticity.
Supported by the Korea National Rehabilitation Center Research Institute (grant no. 2017002), Ministry of Health & Welfare, and Information and Communications Technology R&D program of MSIP/IITP (2017-0-01724).
30246-1&id=fx1.jpg){kind=link}
30246-1&id=gr1.jpg){kind=link}
30246-1&id=gr2.jpg){kind=link}
30246-1&id=gr3.jpg){kind=link}
30246-1&id=gr4.jpg){kind=link}