Reliability and Agreement of 5 Ergo-Kit Functional Capacity Evaluation Lifting Tests in Subjects With Low Back Pain

Participants 

Fourteen physiotherapy (PT) centers in the southern section of Amsterdam were contacted for permission to recruit patients from their practices. All patients were initially contacted by their physiotherapists, who briefly explained the experimental procedures. The patients interested in participating received a folder containing detailed information on the study protocol and were asked to contact the first author (VG). Participant eligibility was determined through telephone interviews, during which potential subjects were asked several questions that were intended to determine whether the subjects met the 3 inclusion criteria: (1) age between 18 and 65 years, (2) had LBP in the last 3 months, and (3) because of LBP, had limited physical capacity in daily activities at home and at work. We defined LBP as 1 or more episodes of pain or stiffness in the low back area within the past 3 months that lasted for a minimum of 7 consecutive days. A power analysisa (confidence interval [CI] method with confidence level of .95, correlation coefficient sets at .90 and limit at .80) indicated that 23 subjects were required for the study. Prior to enrollment, subjects received verbal and written information on the study procedures and signed statements of informed consent. In addition, the subjects were free to withdraw from the study at any time. The study was performed in accordance with the Declaration of Helsinki and was approved by the Medical Ethics Committee of the Academic Medical Centre in Amsterdam.

Ergo-Kit Tests: Selection, Description, and Outcomes 

The standard protocol of the Ergo-Kit assesses 55 subtests, and takes approximately 3 hours to complete. Of the 7 Ergo-Kit physical agility tests concerned with manipulation, balance, strength, and endurance tests that are associated with musculoskeletal complaints, 2 have been shown to be unreliable in adults without musculoskeletal complaints.34 Consequently, we used only 5 of the lifting tests in this study (fig 1). Two were isometric lifting tests: a back-torso lift test (BTLT) and a shoulder lift test (SLT). The other 3 were dynamic lifting tests: carrying lifting strength test (CLST), lower lifting strength test (LLST), and upper lifting strength test (ULST). Table 1 presents Ergo-Kit lifting test descriptions and outcomes. Standardized procedures were performed as described in the Ergo-Kit handbook.17 The Ergo-Kit protocol normally includes 2 tests on the Jamar hand dynamometer, 4 reach tests, and 5 manipulation tests between these 5 lifting tests. The testing order for the 5 lifting tests was not modified.

View Large Image Download to PowerPoint

Fig 1. Five Ergo-Kit FCE method tests: (A) back-torso lift test (BTLT), (B) shoulder lift test (SLT), (C) carrying lifting strength test (CLST), (D) lower lifting strength test (LLST), and (E) upper lifting strength test (ULST).

Table 1. Ergo-Kit Test Descriptions and Outcomes17

	Ergo-Kit Tests	Description	Outcome
	Back-torso lift test (BTLT) Shoulder lift test (SLT)	Use of a “back and leg dynamometer” fixed on a platform, a chain and a handle. Handle is set at patella height for BTLT (fig 1A) and at elbow height for SLT (fig 1B). Maximal pulling during 4 s, 2 tries per test	Maximal isometric lift capacity (kg)
	Carrying lifting strength test (CLST) Lower lifting strength test (LLST) Upper lifting strength test (ULST)	Use of a stand with two vertically adjustable shelves, a box with different weights and a step (20cm). Following standardized procedure, weight is added to the box (2.5, 5, 7.5 or 10 kg), depending on the subject’s coordination in the task, subject’s perception of the weight of the box, and subject complaints. 4−6 carries 5 m for CLST (fig 1C), 4−6 lifts from knuckle height to step for LLST (fig 1D) and 4−6 lifts from knuckle to acromion height for ULST (fig 1E).	Maximal safe weight for lifting (kg)

NOTE. Reprinted with permission.17

Raters 

A list of the 32 available raters in the Netherlands who were certified for Ergo-Kit assessment was obtained from the provider of this FCE method. All had completed the same training program, which consisted of 4 instruction days and at least 12 hours of practice. Because the test assessments were to take place in Amsterdam, selection was limited—for practical reasons—to raters who worked within a 40-km radius of the city. This left 3 raters, 2 of whom were selected at random and agreed to participate. Both raters (R1, R2) had between 4 and 5 years of experience performing the assessments. The raters received financial compensation and travel reimbursement for their participation.

Procedure 

We used a within-subjects design to assess reliability and agreement. Each subject was assessed at 2 different times (t1, t2) by the 2 different raters (R1, R2). Raters assessed all subjects independently and were blinded to the other’s test results. The time interval between t1 and t2 was set at 3 days, as this was considered sufficient to prevent carry-over effects and to give subjects time to recover from the first assessment.36, 37, 38 In addition, each subject was assessed at the same time of the day.39 Subjects were divided into 2 groups, based on their availability, and the raters assessed both groups in counterbalanced order: 1 subject group was assessed at t1 by R1 and at t2 by R2, and 1 group at t1 by R2 and at t2 by R1. Prior to the second assessment, all subjects were asked whether they had recovered satisfactorily from the first assessment. If they had not, they were not allowed to undergo the second assessment, but were permitted to participate in 2 new test sessions at a later date.

LBP: Pain Intensity and Disability 

Before both assessments, the patients were asked to complete an existing Dutch translation of the Von Korff questionnaire40 about their LBP and related disability. This was done to permit us to evaluate whether their health status had changed between t1 and t2. The Von Korff questionnaire has shown a moderate-to-good correlation with other self-reported disability instruments such as the Medical Outcome Study 36-Item Short-Form Health Survey and the Roland-Morris Disability Questionnaire; it has been evaluated as reliable and valid in study samples similar to the one in this study.41, 42 The Von Korff questionnaire assesses pain and disability experienced in the past 6 months; therefore in order to fit our inclusion criteria, we adjusted it to consider only a 3-month prevalence of LBP and disability. Current pain intensity was assessed with 3 questions that were scored on a scale of 0 (no pain) to 10 (the worst pain possible). Disability due to LBP was assessed with 4 questions about the number of days the subjects were disabled and their ability to perform activities and/or work (scored on a scale of 0 to 10). Two total scores were calculated: a 0 to 100 pain intensity score based on the mean of the pain intensity questions multiplied by 10, and a 0 to 100 disability score based on the mean of the disability questions multiplied by 10.40

Kinesiophobia and LBP 

Subjects were asked to fill in the Dutch version of the Tampa Scale of Kinesiophobia (TSK)43 to assess their fear of reinjury caused by physical movement and activity. The TSK covers 17 items, each of which is scored on a 4-point Likert scale ranging from “strongly disagree” to “strongly agree.” For each subject, a total score ranging from 17 to 68 was calculated after inversion of the individual scores for items 4, 8, 12, and 16. The TSK has shown good reliability and validity in different study populations.44, 45 The TSK was completed after each test session to avoid eventual effects on subjects’ performance provided by the assessment of this questionnaire.

Data Analysis 

Means, standard deviations (SDs), and ranges were calculated for each test for raters 1 and 2. The level of reliability was expressed with an intraclass correlation coefficient (ICC)20, 46, 47 and determined with the test scores assessed by the 2 raters. We used the ICC model 2.1.A, based on a mixed 2-way analysis of variance, as defined by Shrout and Fleiss.48 The 95% CI was calculated for each ICC mean. The ICC and 95% CI values were evaluated as follows20, 26, 49, 50: “low” reliability when ICC means and/or CI lower bounds were lower than .50; “moderate” reliability when ICC means and/or CI lower bounds ranged from .50 to .80; and “high” reliability when ICC means and CI lower bounds were greater than .80.

To assess the raters’ stability in repeated measurements over time and to gain an insight into the clinical relevance of the Ergo-Kit lifting tests, agreement was expressed with the standard error (SE) of measurement (SE of measurement = √[var(raters) + var(error)] or SE of measurement = SD × √[1 – ICC]) and its 95% CI (95% CI = 1.96 × SE of measurement).20, 33 Using a general linear model, we calculated 3 different components of variation, variance between subjects (var[subjects]), variance between raters (var[raters]), and variance due to measurement error (var[error]). To explore the stability of the patients’ health status from 1 test to another, their mean scores for LBP pain intensity and related disability (disability score), and kinesiophobia were calculated from the Von Korff questionnaire and TSK at t1 and t2. For these 3 variables, statistical differences between t1 and t2 were explored with paired t tests. All analyses were performed with the statistical analysis software SPSSb for Windows.

Participant Characteristics 

Twenty-five subjects with LBP (11 men, 14 women) were recruited for this study. All subjects were working either parttime or full-time in a variety of professions. One subject was not able to perform the second test assessment because he did not recover properly from the first test. The subjects’ mean age ± SD was 49±8 years (range, 34–63y), their mean height was 175cm (range, 158–195cm), and their mean body weight was 78kg (range, 48–97kg). There were few differences between t1 and t2 in terms of the subjects’ LBP pain intensity (P=.003) and related disability, and their subjects’ TSK mean scores (table 2). These small differences in average pain intensity, average disability, and average TSK scores, however, do not appear to be clinical relevant changes within subjects from 1 test session to another.

Table 2. Scores and Observed Differences of LBP Pain Intensity, Disability, and Kinesiophobia

	Items	N	t1	t2	|D|⁎	P
	Pain intensity (range, 0−100)	24	62.7±19.9	56.2±19.1	6.5	.003
	Disability score (range, 0−100)	24	46.4±29.5	41.0±26.0	5.4	.141
	TSK total score (range, 17−68)	24	39.3±6.7	40.2±6.2	0.9	.369

NOTE. Values are N or mean ± SD.

⁎ Absolute difference between t1 and t2.

Reliability 

Table 3 presents the averages, SDs, and ranges in scores for all 5 Ergo-Kit tests for both sessions assessed by the raters, their mean ICCs, and corresponding 95% CIs. The level of interrater reliability was high for both isometric strength tests (BTLT, SLT), as their mean ICCs were .97 and .96, respectively, with CI lower bounds of .94 and .91, respectively. The mean ICCs for the 3 dynamic strength tests were .95 for the CLST and the ULST, and .94 for the LLST. The corresponding CI lower bounds (.84, .89, and .85, respectively) are considered highly reliable.

Table 3. Test Scores Assessed by R1 and R2 (time interval, 3d) and Interrater Reliability Levels

	Tests (kg)	N	Rater 1 (s)		Rater 2 (s)		ICC	ICC 95% CI
	BTLT	24	65.9±38.3	20.5–180.5	63.3±39.5	12.5–177.5	.97	.94–.99
	SLT	24	37.6±18.3	14.0–72.0	38.9±19.1	10.0–76.5	.96	.91–.98
	CLST	24	24.5±9.7	10.0–47.5	22.1±11.2	7.5–47.5	.95	.84–.98
	LLST	24	23.8±11.1	7.5–47.5	21.8±10.6	7.5–47.5	.94	.85–.97
	ULST	24	17.0±6.3	7.5–32.5	17.1±6.6	7.5–30.0	.95	.89–.98

NOTE. Values are N, mean ± SD, and range.

Variation Components 

The variation components (between subjects, between raters, systematic error) for all 5 tests are presented in table 4. Given the ratio between all 3 variation components in all 5 tests, var[raters] is relatively small, whereas var[subjects] is relatively high.

Table 4. Variation Components and Indicators of Agreement per Ergo-Kit Test

	Tests (kg)	N	Var[subjects]	Var[raters]	Var[error]	SEM	SEM 95% CI
	BTLT	24	1444.90	0.36	72.66	8.6	X±16.7
	SLT	24	324.65	−0.16⁎	25.06	5.0	X±9.8
	CLST	24	101.40	2.50	8.83	3.4	X±6.6
	LLST	24	105.92	1.68	11.87	3.7	X±7.2
	ULST	24	37.55	−0.16⁎	3.93	1.9	X±3.8

Abbreviations: SEM, standard error of measurement; X, observed test score.

⁎ Negative variance components are treated as though they are zero.

Agreement 

Table 4 presents the SE of measurement and the 95% CI for each test, which offers a clear picture of the agreement between the raters. The SE of measurements, expressed in kilograms, are small, especially given the mean values of the different tests (see table 3). For instance, the BTLT mean score from both test assessments approaches 64kg, its SE of measurement 8.6kg, and its CI 47 to 81kg. This indicates that an increase or decrease of 17kg from the observed score cannot be interpreted as a change resulting from a measurement error.