| | Effectiveness Evaluation of a Remote Accessibility Assessment System for Wheelchair Users Using Virtualized RealityPresented to the Rehabilitation Engineering Society of North American, June 2006, Atlanta, GA. Abstract Kim J, Brienza DM, Lynch RD, Cooper RA, Boninger ML. Effectiveness evaluation of a remote accessibility assessment system for wheelchair users using virtualized reality. ObjectiveTo determine the value of the Remote Accessibility Assessment System (RAAS), a 3-dimensional (3D) image reconstruction technology designed to analyze accessibility of the target built environment in a virtualized reality, in assessing a built environment’s accessibility by calculating the congruence level between the RAAS and conventional in-person method. DesignRepeated-measures (within-subject) design. SettingA university research laboratory. ParticipantsThree homes for people who use wheeled mobility devices. InterventionHome physical environment was divided into several potential problem areas such as entrance, hallway, bathroom, and living room. Each area was identified by several tasks that might be performed in it. All possible tasks in each area within each home were evaluated using 2 methods: RAAS and the conventional in-person assessment. The evaluations were performed by a different home modification specialist for each method. Main Outcome MeasuresConventional in-person assessments were cross-tabulated with assessments from RAAS, with which there are 4 possible assessment combinations. A true positive (checked–checked) occurs when the RAAS method checks the target task as problematic and it is also checked as problematic by the conventional in-person method. True negative (not checked–not checked), false positive (not checked–checked), and false negative (checked–not checked) were also identified as the same way. ResultsThe proportion of overall agreement was high at 94.1% and the overall sensitivity and specificity was 95.6% and 90.3%, respectively. A significant κ coefficient of .857 and the 95% confidence interval of the odds ratio of 104.062 to 404.921 were calculated and a high level of overall agreement rate was shown. A high P value (.868) of the McNemar test implied that there was no marginal homogeneity, that is, no tendency to identify the task incorrectly in the positive or negative direction. ConclusionsThis system proved that virtualized reality and 3D reconstruction technology may provide an effective means to investigate the architectural features of a built environment without an expert visiting the site. This system could become an efficient tool for the service provider and can provide expert service to underserved clients that would otherwise be unavailable. ACCORDING TO THE U.S. Census Bureau’s Survey of Income and Program Participation, the number of wheelchair users aged 18 years or older in 1999 was estimated at more than 2.3 million in the United States.1 An important trend in usage of wheeled mobility devices is that the number of people using wheelchairs is increasing; thus the demand is likely to continue to grow.2 For any given limitation in function, the amount of disability that a person experiences will depend on the quality of the social and physical environment.3 Consideration of the built environment is especially critical for wheelchair users, given the potential limitations that the environment can impose. Most important, for mobility devices to be used effectively, the environments in which they are used must be physically accessible.4 The 1995 American Housing Survey asked whether members of households had permanent physical activity limitations and, if so, whether home modifications had been performed. Based on this survey, approximately 5.1 million (57.4%) of the households in which a member had an activity limitation had no home modification.5 Struyk and Katsura,6 in their extensive study of the modifications made by older persons in order to remain in their homes, addressed a number of obstacles standing in the way of securing modifications for those who need them, such as: (1) unclear policy responsibilities; (2) inadequate and medically based reimbursement programs; (3) lack of adequate environmental assessments; (4) reluctance by older adults themselves to change their environment; and (5) an undeveloped service delivery system. The home environment introduces considerations related primarily to safety and the performance of basic living activities.7 Home modification has come to be recognized as an important intervention strategy to manage health care conditions, maintain or improve functioning, ensure safety, and reduce the wheelchair user’s dependency on others.8 Effective home modification requires consultation with skilled professionals capable of assessing the home environment and identifying changes necessary to meet the wheelchair user’s needs. Although there are many building and remodeling contractors able to perform the modifications, the availability of skilled professionals with experience in home modifications for accessibility is limited. Providing services in rural areas is particularly difficult. Such service requires lengthy travel times that increase cost and consume the limited time of skilled professionals. Other Studies  A system that enables accurate remote assessments would be an important tool to improve our ability to perform home assessments more easily and possibly at decreased cost. This study addressed the development and evaluation of a remote accessibility assessment system using virtual reality technology. Some developmental work has been done by others using remote assessment systems in rural or underserved areas. A team of clinicians at the Shepherd Center (Atlanta, GA) performed a case study of remote home modification evaluation using a videoconference system with a video telephone.9, 10 They showed that remote telerehabilitation assessments have the potential to enable specialists to diagnose potential accessibility problems in home environments and to prescribe appropriate modifications regardless of the location of the client, home, or specialist. Another effort was undertaken by Extended Home Living Services (Wheeling, IL), which developed a remote assessment protocol using a survey instrument, the Comprehensive Assessment Survey Process for Aging Residents (CASPAR). The CASPAR instrument can be mailed to residents in remote areas to obtain information about consumer’s priorities, their activities of daily living, their ability to participate in home-specific occupations, and the space, layout, and design of their residences, so that home modifications can be recommended.8, 11 However, both of these studies are limited in that the dimensions obtained were not sufficient for use in specifying modifications. Both methods depended on dimensions obtained by a layman, using a tape measure. The Shepherd Center’s research team used a video conferencing system transmitting through a conventional land phone line, but the low bandwidth of the land line could not provide sufficient resolution to discern the physical objects in detail. Moreover, in addition to the services of a home modification specialist, the study required a technician skilled at operating video equipment, who would be paid as much for travel and labor as would the home modification specialist. This additional expense might threaten the cost-effectiveness of the intervention. The CASPAR had another limitation: with no 3-dimensional (3D) view of the structure of the built environment, it depended on photographs taken by customers. Virtual Reality and 3D Reconstruction The term virtualized reality was coined and introduced in a paper by Kanade et al.12 The traditional virtual reality world typically is constructed using simplistic, artificially created computer-aided design models. Virtualized reality starts with the real-world scene and virtualizes it. We have entered an era in which the acquisition of 3D data is ubiquitous, continuous, and massive. One of the best areas to which 3D reconstruction of real-world objects and scenes can be applied is the architectural environment. As technology advances in laser-scanning techniques, 3D modeling software, image-based modeling techniques, computer power, and virtual reality, 3D reconstruction of cultural heritage applications using digitization and modeling has become increasingly common.13 A primary goal of computer vision is the reconstruction of 3D shape from 2-dimensional (2D) visual images. Although active methods such as range finding or laser striping are accurate, they require expensive equipment. The problem of cost has motivated work toward the implementation of passive techniques that seek to infer 3D depth information from 1 or more 2D images.14 Photogrammetry, which loosely translates from the Greek as “light drawn to measure,” is the technique of obtaining measurements from photographs, and it can provide a cost-effective alternative. The use of engineering photogrammetry to achieve extremely accurate 3D models has become affordable and convenient with improvements in the processing power of desktop computers and the ready availability of inexpensive, user-friendly packages for image processing.14 Remote Accessibility Assessment System Using Virtualized Reality Therefore, the use of virtual reality technology and telerehabilitation concepts to assess the home built environments of persons with severe mobility impairments was proposed.15, 16 A Remote Accessibility Assessment System (RAAS) using virtual reality was developed as part of this project. The RAAS was designed to evaluate the accessibility of physical environments of wheelchair users, using a virtualized 3D model. The RAAS takes advantage of state-of-the-art digital imaging, 3D reconstruction, and photogrammetry technologies. The outcome of the assessment depends on measurement accuracy, which depends on the skill of the person who takes the measurements. The RAAS can potentially overcome limitations of previous studies by providing an accurate measurement tool and by allowing the evaluation specialist, architect, or rehabilitation engineer to see the space in 3D scenes. This study could produce better results than have previous studies because specialists can evaluate the environment with more realistic, visual information beyond numeric data. Nevertheless, accuracy remains a critical concern in the virtualized environment17 and usability is a primary concern for the telerehabilitation system.18 Accuracy and usability are thus keys to developing a successful system. Preliminary Studies We performed 2 reliability analyses on the hardware and software components: (1) verification that the commercial software, Photomodeler Pro,a can construct sufficiently accurate 3D models by analyzing the accuracy of the dimensional measurements in the virtualized environment; and (2) comparison of dimensional measurements with 4 camera types. Based on these 2 analyses, we were able to specify a consumer level digital camera and the Photomodeler Pro software for this system.19 In the third phase, we tested the system in an actual environment to evaluate its ability to assess the accessibility of a wheelchair user’s typical built environment. This feasibility test resulted in an accurate accessibility assessment, thus validating our system.20 Through these pilot studies, algorithms for constructing 3D models of wheelchair user’s home environments and for assessing the environment’s accessibility were developed, including the development of several new tools, including a guidelines book on how to take pictures, a survey form, a measurement form, and an evaluation form.21 Objective As the last phase of our study, we evaluated the capability of the RAAS, as compared with that of the conventional in-person method, in assessing the accessibility of the wheelchair user’s built environment. The objective of this study was to determine the value of the RAAS in assessing a built environment’s accessibility by calculating the congruence level between the RAAS and conventional in-person method.22, 23 Methods Research Methodology This study used a repeated-measures research design. We divided a home physical environment into several potential problem areas such as entrance, hallway, bathroom, and living room. Each problem area was identified by several tasks that might be performed in it. All possible tasks in each area within each home were evaluated using 2 methods: RAAS and conventional in-person. In this within-subject design, tasks evaluated by the RAAS method would be the experimental group and the same tasks evaluated by the conventional in-person method would be the control group. Two architects who specialize in accessibility participated in this study as evaluators. One architect has worked for architectural barrier removal, barrier-free design, and universal design for over 34 years. He has been involved in research projects and lecturing and teaching at university level. He was appointed to represent the national organization of United Cerebral Palsy Associations on the American National Standards Institute (ANSI) A117 Committee on Accessible and Useable Buildings and Facilities of the ANSI, and has continued as a voting, standards-writing member for the past 19 years. The other architect was an employee of the architectural firm of the first architect in the early 1990s. She learned much about accessibility under his employ and tutelage, and she went on to found her own firm. Her architectural practice progressed much like his own, and she has also become a recognized expert in the field of accessible design. The evaluators were blinded to each other’s assessment. The assessment addresses several potential problem areas of the home, and each area has a number of associated tasks. Each task was designated as problematic or not, and hence, in need of modification or not, by each architect evaluator. In this study, 3 houses were evaluated for 3 exemplar subject situations and 6 problem areas were identified for the evaluation of each house. A number of objects (usually ≈70) were evaluated in a home evaluation by both evaluators using the same evaluation form. We adapted some of the tasks of the CASPAR as checklist items of our developed evaluation form. In addition to the tasks of the CASPAR, we added some features necessary to wheelchair users, such as whether enough space exists to build a ramp or to install a stair glide or a lift. Participants The actual case of the recruited client was not evaluated for comparison by the 2 methods because of differing information. The architect from Lynch and Associates had preliminary information about his clients through interviews and discussions with them; however, the other architect for the RAAS had no preliminary information about the clients. This fact can severely threaten the internal validity of the research protocol. Instead of an actual client, 3 exemplar subjects were created. The first subject, who was injured at T11 in a car crash, uses a manual wheelchair. The second is a power wheelchair user with a diagnosis of spastic quadriplegia cerebral palsy. The third exemplar was diagnosed with multiple sclerosis 5 years ago. He uses a scooter as his primary means of mobility and he can walk a short distance. The information about diagnosis and mobility aids of the imaginary subjects were filled out in the survey form by the investigator and provided to both evaluators. We selected these 3 diagnoses and 3 types of wheeled mobility device because we considered them to be appropriate diagnosis and wheeled mobility devices, which represent the target population well, on the experience at the wheelchair clinic of our department. Procedures Recruitment If a person requested a home accessibility assessment through the architecture firm Lynch and Associates, a flyer was distributed to him/her by the architect. The architect then instructed any interested client to contact an investigator of this study. Clients who were interested in the study were contacted by the investigator by telephone. At this time, the investigator discussed the study and its risks and benefits in further detail. A formal consent form and self-addressed stamped envelope were then mailed to potential participants, accompanied by a cover letter instructing the person to contact the investigator by telephone before signing the informed consent so that any questions he/she might have about the study could be addressed before consent was given to participate. We recruited 3 houses from clients who requested the accessibility assessment by Lynch and Associates. We got the approval of University of Pittsburgh Institutional Review Board (IRB) in order to take pictures of these home environments (IRB no. 0406073). One house was a ranch style house, another was a multilevel house, and the third was a 3-story Victorian style house. Creating exemplar subjects Instead of actual clients, 3 example living situations were created through completion of the survey form for 3 exemplar subjects. These survey forms were filled out by the investigator for each of 3 exemplar subjects for each house. Acquisition of images Pictures were then taken of each example home environment. In this study, 3 students were recruited as part-time assistants for the image acquisition. They were instructed in how to take pictures of the home environment with the developed manual. Each of them was sent to one of the homes with a Canon A60 digital camerab and 4 carpenter’s squares. When the student took pictures of a problem area, at least 2 squares were located in the middle of the space in order to provide information to verify that the accuracy of the model is high enough for use in the RAAS method. We used the carpenter’s scale, the Empire Steel Carpenter Square,c and an aluminum framing square, as the normative objects. Then this normative object also could be used for calibration of the space. The student also sketched a rough floor plan of the home environment that showed where each problem area was located. Evaluation through conventional in-person The architecture firm of Lynch and Associates conducted the conventional in-person assessment by visiting their client’s home and investigating the physical environment. An architect and an architectural assistant measured the architectural dimensions with tape measures. The architect then examined all tasks on the evaluation form for each exemplar subject and determined what tasks presented problems and what modifications were needed. The architect completed the evaluation form with the information from the on-site investigation and measurement of his/her client’s home environment. This procedure was performed before the acquisition of images. 3D reconstruction The pictures were sent back to the research center with the sketched floor plan. A technician—who is an investigator in this study—analyzed the pictures and constructed 3D models. A 3D model was made for each problem area, using the 3D modeling software, Photomodeler Pro. Once the model was constructed, the technician measured the dimensions of the normative objects (carpenter’s squares) in the 3D model to determine whether the model was accurate enough to be used for the accessibility assessment. We analyzed the accuracy of the 3D models by calculating the deviation between actual known dimensions of squares and the dimension measured in the 3D model and comparing the deviation with the tolerable level. If the deviation of any measured dimension from the normative object was larger than the agreed accuracy level (1:30), the technician had to refine the model until it was sufficiently accurate. One way to improve the accuracy is to mark the object points more clearly in Photomodeler. If we could not obtain sufficient accuracy by remarking, we improved the accuracy by adding or changing the pictures used to create the model. Evaluation through the RAAS Once the 3D models were created with the desirable level of accuracy, that is, all measurements of normative objects were within the agreed accuracy level (1:30), they were given to another architect along with the 2D photos and a sketched floor plan. The survey form of each exemplar client was also provided with the evaluation form for each case. Another architect then evaluated the accessibility and assessed the modification requirements for each exemplar client’s situation, using the virtualized model of each home environment, and referring to 2D photos and preliminary information from the survey form (fig 1 shows an evaluator working with the RAAS method). We also used the evaluation form to evaluate all tasks in all problem areas in an orderly and systematic way. The evaluation form was then used to compare the 2 evaluations, one by the RAAS method and the other by the conventional in-person method. Comparing 2 methods As described above, in order to compare the 2 methods, each exemplar client’s situation in the chosen home environment was assessed through conventional in-person by the architect at Lynch and Associates and through RAAS by another architect from Tusick and Associates. The first architect used the floor plan of each house that was made by on-site measurement and the second architect used 3D models to evaluate the accessibility. The investigator compared the data from each evaluation form, completed by the 2 architect evaluators, to determine the level of agreement between the evaluation results through the RAAS method and the conventional in-person method. This field evaluation was performed for 9 cases from 3 exemplar subjects and 3 home environments. Data Collection and Analysis To analyze the degree of conventional in-person and/or RAAS agreement, conventional in-person assessments can be cross-tabulated with assessments from RAAS, in which there are 4 possible assessment combinations. A true positive (checked–checked) occurs when the RAAS method identifies the target task as problematic and it is also identified as problematic by the conventional in-person method. A true negative (not checked–not checked) occurs when the RAAS method identifies the target task as nonproblematic and it is also identified as nonproblematic by the conventional in-person method. A false positive (not checked–checked) occurs when the RAAS method identifies the target task as problematic but it is identified as nonproblematic by the conventional in-person method. A false negative (checked–not checked) occurs when the RAAS method identifies the target task as nonproblematic but it is identified as problematic by the conventional in-person method. When an evaluator identified a task as nonproblematic, that does not mean he evaluated it correctly or incorrectly. Each evaluator can evaluate a task as problematic or nonproblematic, but we cannot say who evaluated correctly. We want to see if they evaluated it with the same result or with different results. Therefore, we analyzed the congruency rate between results of 2 evaluation methods. These assessments depend on the experience and perception of the evaluators. To avoid the bias and variability that could occur from difference of performance between evaluators, 2 architects who are very experienced in home modifications and accessibility assessments for people with disabilities participated in this study as evaluators. They are both very renowned architects in the Pittsburgh area as home modification specialists for the accessibility of people with disabilities.24 We developed an evaluation form (fig 2) by adapting some of the tasks of the CASPAR as checklist items and added some features necessary to wheelchair users, such as whether enough space exists to build a ramp, to install a stair glide, or to install a lift. The contents of the evaluation form are consistent with the physical environment part of the survey form. That is, the contents are divided into several problem areas and each area contains several tasks. This form allowed evaluators to easily identify potential problematic tasks and guided them through a systematic assessment, reducing the bias between evaluators and allowing us to analyze the agreement rate between 2 methods objectively. In this study, each home had 6 areas and each area had about 10 tasks. A total of 612 tasks were evaluated for 9 cases. Table 1 shows what areas were investigated in each home environment and how many tasks were evaluated in each area. Table 2 shows the cross-tabulated data from this field evaluation. We collected 9 samples of this type of dichotomous datum and analyzed them with so called “rater agreement” methods.25 The generic word rater is used here to refer to the accessibility evaluation method. Results Overall Agreement Rate We used the conventional in-person assessment as the baseline to compare the RAAS protocol. The proportion of overall agreement was high at 94.1% and the overall sensitivity and specificity was reported as 95.6% and 90.3%, respectively (table 3). As a significant κ coefficient of .857 and the 95% confidence interval (CI) of the odds ratio (OR) of 104.062 to 404.921 were calculated, a high level of overall agreement rate was shown. And high P value (.868) of the McNemar test implied that there was no marginal homogeneity, that is, no tendency to identify the task incorrectly in the positive or negative direction. Agreement Rates by Home, Subject, and Area Problems were identified in 3 home categories and 3 subject categories. The percentage of correctly identified problems within each category was higher than 90% across all categories (see table 3). All κ coefficients also were larger than 0.7 and all ORs were large, indicating that the agreement rates between 2 assessment methods were very high across all home and subject categories. And all P values of the McNemar test showed that there was no tendency that identifies problems falsely either in the positive or in the negative way. Five problem areas were evaluated across 9 cases, which are cross-matched by 3 houses and 3 exemplar subjects (table 4). The problem areas were getting in/out of the home entrance, getting up and down interior stairs, moving around the house, using the bathroom, and using the kitchen. As for sensitivity, the percentage of correctly identified problems within each area category was above 92% and the correct identification rate at getting up and down interior stairs was 100%. Specificities at 3 areas were 100% and only 1 negative agreement rate involving the “using the bathroom” was notably low, 65.2%. Inversely, a false positive rate for using the bathroom was relatively high, at 34.8%. Therefore, the P value of the McNemar test at using the bathroom was very small, .004, implying a tendency to incorrectly identify as problematic a task that was otherwise identified as nonproblematic. This indicates that the evaluator at the RAAS is likely to assess more conservatively than the conventional in-person evaluator. Time for Constructing 3D Models For the RAAS, the construction of a 3D model is a fundamental requirement. The analysis of consumed time could provide us some estimation on cost-effectiveness even if we could not analyze the exact cost. It took 1 to 3 hours to construct a 3D model of a problem area. The more complex the structure, the more time consumed. The back entrances of the second and third houses were simple. Only 3 photos were used to construct 3D models and it took 1 hour. The hallway of the first house was T-shaped and had many doorways as we can see in figure 3. It required 14 photos and 3.5 hours to construct a model. On average, we estimate that 2 hours were needed to construct the 3D model of a single area and 12 hours for a whole house (table 5). Agreement Rates by Individual Task In this study, 42 tasks were evaluated across 9 cases and 5 critical areas. The number of assessments for each task varied from 3 to 39, and overall, 612 task assessments were performed for 3 houses and 3 subjects. There was 100% agreement on 26 of 42 tasks. A false assessment was detected only on 16 tasks. Four tasks were chosen as examples: A01: Having enough space to build a ramp, A06: Going through the entry door, D05: Reaching or using toilet paper, and F01: Maneuvering space at one of the cabinets. The agreement rates of these tasks were recorded in table 6. Although these 4 sample tasks among 42 tasks have relatively large numbers (A01: n=12, A06: n=39, D05: n=15, F01: n=9) compared with other tasks, the sample size was still rather small, and the calculated κ coefficients were low and inconsistent (A01: κ=.636, A06: κ=.235, D05: κ=.595, F01: κ=.067). Moreover, P values of κ coefficients failed to achieve statistical significance (P>.001). ORs also showed low level CIs that could include 1. This means that there is no significant evidence to support the agreements between assessments by 2 different methods. Therefore, in order to determine which tasks have high agreement, we need to widen the study area to include more diverse houses and clients. Accuracy of Reconstructed 3D Models To assess the accuracy of the constructed 3D models, we investigated the accuracy level of the actual physical objects in the reconstructed 3D models. When the 3D model was created, the accuracy of each model was evaluated by comparing the dimension of the normative objects with their measurements in the model. All deviations between the real dimensions and measurements in each 3D model of the normative objects were within 2.54cm (1in) when we constructed the 3D models. Therefore, we proceeded to the RAAS evaluation procedure using these reconstructed 3D models of the target problem areas. After the RAAS evaluation was conducted, we determined the accuracy by comparing the real object’s measurements between 2 methods in order to confirm the accuracy of the models. We compared 2 measurements of a dimension in each of 18 problem areas between in 3D models and in real spaces. As shown in table 7, this retrospective analysis also showed that the deviations between measurements in the model and by on-site tape measure were within the tolerable level, 2.54cm or so. Discussion Agreement Rate This study showed an overall high agreement rate of 94.1% and high sensitivity and specificity (95.6% and 90.3%, respectively). These results suggest that the RAAS offers a viable alternative for home modification specialists to provide accessibility assessment without the need to perform an on-site assessment. It must be noted that given the design of the study it would not be possible to completely eliminate any intrarater or intrahome correlation. Architects, whether using conventional in-person or RAAS, are likely to look at an area of a house in a global way. Furthermore, it would be impossible to completely eliminate any dependence between tasks. However, the evaluation form was designed so that architects were given the opportunity to evaluate each task within an area independently of all others. Architects were not asked to evaluate the accessibility of a given area for a single item, but rather to separately evaluate several tasks related to that area. For example, in the evaluation form shown in figure 2, it can be seen that the architect evaluated some of the tasks within the entrance to be a problem, but other tasks within the same area not to be a problem. Each architect separately evaluated a total of 612 tasks (see table 1), and the agreement between evaluations of architects on 612 tasks contributed to the overall agreement between the 2 methods. Therefore, intraperson and intrahome correlations should not compromise the validity of statistical results. Comparison With 2 Previous Studies The data from this study compare favorably with the results previously reported for the CASPAR, a remote, paper and pencil assessment protocol.8 They reported 74% sensitivity for problems identified and 92.8% specificity. And the telerehabilitation system using the telephone line based videoconferencing system reported the overall true response rate of 87.1% with sensitivity (86.4%) and specificity (88.2%) of the remote instrument.10 The RAAS provides specialists with 3D views of the physical environment and photos oriented with a 3D model, which gives specialists the opportunity to better determine the environment and to more easily measure the physical 3D dimension. These features might contribute to the improved performance of this study. This study took advantage of instruments developed by previous studies such as the CASPAR and tried to overcome the limitations of those studies, such as inaccurate measurements. Exemplar Subjects We created exemplar subjects in order to more accurately compare the 2 evaluation methods for 3 houses. In an actual application, the evaluation results would depend on the client’s information, because a study has shown some older adults themselves are reluctant to change their environment.6 For example, the client might provide the information that his/her hallway presented no problems and thus did not need to be evaluated. But his/her opinion might not be correct, which might then prevent the assessment of a potentially problematic area. Conversely, too much preassessment information could threaten the objective ability of this research to compare the assessments by 2 methods. Therefore, in order to evaluate the new method objectively, we decided to create exemplar subjects instead of using an actual client. In this study, the evaluator using the conventional method could perform the on-site investigation and could call back any time to question the house owners or occupants about the built environment, but the evaluator using remote protocol could not contact the owner or occupant. To compensate, the RAAS evaluator was able to address questions about the built environment to the investigator who analyzed the photos in detail and made 3D models and to the student assistant who visited the house and photographed and sketched the floor plan that would be referenced for the RAAS evaluator in order to understand the whole structure of the home. They answered the questions to the best of their abilities. But in the actual application, the service provider using the RAAS could contact the house owner or occupant by telephone or video conferencing system to get more information about the built environment. It is expected that when RAAS is implemented in a real world situation, both the person who administers the assessment and the client who is assessed will be contacted with questions that arise from analysis of data in RAAS and will be part of the assessment decision process just as they would be in a conventional on-site assessment. Time Consumption of the RAAS and Conventional In-Person Evaluation Methods In order to evaluate a target environment, we needed to take 2D pictures, create 3D models, and analyze the environment in the virtualized reality with 2D photos. The architect who evaluated the home by conventional in-person also took pictures in order to analyze the home later in his office. Although the 3D reconstruction with 2D photos require much technician time, the conventional in-person method required at least a day of travel time and measuring time by an expert architect and an assistant. As for analyzing the environment and evaluating the accessibility on the table of the architect’s office, it took similar time for both methods. Therefore the RAAS has a cost effective merit compared with the conventional in-person evaluation. Moreover, we can value the critical advantage of the RAAS because affordability of service delivery is more important than cost effectiveness for clients in underserved areas. In particular, we noted that the greater the geographical distance, the greater the benefits of the RAAS method. Generalizability We evaluated the houses of 3 clients who requested the accessibility assessment by the architecture firm, Lynch and Associates. We created 3 exemplar subjects who had different diagnoses and who used different wheeled mobility devices. We tried different situations in order to test the external validity. Although the sample size of houses and subjects was small, this study showed that the RAAS might be applied to underserved areas due to lack of available experts within the region. However, to generalize this study more, we need to expand our study into more diverse kinds of houses, diagnoses, and wheeled mobility devices, with larger sample sizes, so that our developed system could be applied to other underserved areas including available experts in the region. Study Limitations Although we showed the potential value of the RAAS through field trials, the method has limitations. Even with the guidebook, it is a challenge for novices to take appropriate 2D pictures for the 3D reconstruction. In a follow-up to this study, we have designed a new protocol that uses videoconferencing through a high-speed Internet connection that allows the consumer to confer with the specialist while photographing the environment. Using this new system, the provider can guide the consumer through the picture taking process, thereby ensuring the inclusion of important features of the environment. Once appropriate photos are taken, the process is limited by complex photogrammetry software that is time consuming and difficult to learn. Newer software is easier to use and lower cost. Automatic 3D reconstruction technologies using a camcorder or laser scanner should also be considered for future studies. RAAS cannot provide sufficient and effective communication between the consumer and service provider. Using the same evaluator for 1 method is a weakness that could have biased our results. One evaluator assessed 3 houses using the RAAS and the other performed only the conventional in-person method. The nonrandom assignment of evaluators was necessary because the subjects were clients of the architect who participated in the study. Another limitation of this study and possibly of the remote assessment is the lack of involvement by a clinician in the evaluation process. Such involvement may lead to better understanding of the client’s unique needs, their functional limitations related to inappropriate environments, and identification of additional ways to improve the physical environment relative to their abilities. In addition, although use of imaginary subjects is a reasonable approach to this study, the evaluator was unable to directly assess patient-environment interactions. Finally, a small sample size may limit the generalizability of our results. We are planning larger randomized studies to address this limitation. Conclusions This study showed a high agreement between the conventional in-person method and the RAAS. Findings suggest that the RAAS assessments have the potential to enable specialists to assess potential accessibility problems in built environments regardless of the location of the client, home, or specialist. Most important, this system proved that virtual reality and 3D reconstruction technology may provide an effective means to investigate the architectural features of a built environment without an expert visiting the site. Even if 3D reconstruction requires cumbersome trials and comprehensive manipulation of the software and the involved photographing process, the protocol could be improved continuously by adapting state-of-the-science technologies such as remote photographing, video-based 3D modeling, and laser scanning. This system can become an efficient tool for the service provider and can provide expert service to underserved clients that would otherwise be unavailable. The developed virtual reality telerehabilitation system could improve rehabilitation outcomes by making accessibility assessments and modifications available to a larger portion of the population of wheelchair users. Suppliers Acknowledgments We thank Susan Tusick, of Tusick and Architect Associates, for her participation as an evaluator and Elaine Rubinstein, PhD, of Office of Measurement and Evaluation of Teaching, University of Pittsburgh, for her statistical advice. References 1. 1McNeil JM. Prevalence of disabilities and associated health conditions among adults—United States. MMWR Morb Mort Wkly Rep. 1999;50:120–125. 2. 2Cooper RA, Cooper R. Trends and issues in wheeled mobility technologies. In: Space Requirements for Wheeled Mobility Workshop: An International Workshop; 2003 Oct 9-11; Buffalo (NY). Available at: http://www.ap.buffalo.edu/idea/Space%20Workshop/index2.asp. Accessed Oct 25, 2007. 3. 3In: Brandt EN, Pope AM editor. Enabling America: assessing the role of rehabilitation science and engineering. Washington (DC): Natl Acad Pr; 1997;. 4. 4LaPlante LM. Demographics of wheeled mobility device users. In: Space Requirements for Wheeled Mobility Workshop: An International Workshop; 2003 Oct 9-11; Buffalo (NY). Available at: http://www.ap.buffalo.edu/idea/Space%20Workshop/index2.asp. Accessed Oct 25, 2007. 5. 5U.S. Department of Housing and Urban Development, U.S. Census Bureau. American housing survey for the United States, supplement. Washington (DC): U.S. Government Printing Office; 1995;. 6. 6Struyk RJ, Katsura H. Aging at home: how the elderly adjust their housing without moving. J Housing Elder. 1987;4:1–192. 7. 7Pope AM, Tarlov AR. Disability in America: toward a national agenda for prevention. Washington (DC): Natl Acad Pr; 1991;. 8. 8Sanford JA, Pynoos J, Tejral A, Brown A. Development of comprehensive assessment for delivery of home modifications. Phys Occup Ther Geriatr. 2002;20:43–55. 9. 9Burns RB, Crislip D, Daviou P, et al. Using telerehabilitation to support assistive technology. Assist Technol. 1998;10:126–133. MEDLINE 10. 10Sanford JA, Jones M, Daviou P, Grogg K, Butterfield T. Using telerehabilitation to identify home modification needs. Assist Technol. 2004;16:43–53. MEDLINE 11. 11Pynoos J, Sanford JA, Rosenfelt TA. Team approach for home modification. AOT Pract. 2002;159:. 12. 12Kanade T, Narayanan PJ, Rander PW. Virtualized reality: concepts and early results. In: IEEE Workshop on the Representation of Visual Scenes; 1995 June 24; Boston (MA). p 69-76. 13. 13Ponce J̇, Genc Y, Sullivan S. Constructing geometric object models from images. In: IEEE International Conference on Robotics and Automation; 2000 Apr; San Francisco (CA). p 871-8. 14. 14Pratt T. From photo to 3D model. IEEE Rev. 2000;46:9–12. 15. 15Cooper RA, Fitzgerald SG, Boninger ML, et al. Telerehabilitation: expanding access to rehabilitation expertise. Proc IEEE. 2001;89:1174–1193. 16. 16Cooper RA, Ding D, Simpson R, et al. Virtual reality and computer-enhanced training applied to wheeled mobility: an overview of work in Pittsburgh. Assist Technol. 2005;17:159–170. MEDLINE 17. 17Durlach NI, Mavor AS. Virtual reality: scientific and technological challenges. Washington (DC): National Research Council; 1995;. 18. 18Lathan CE, Kinsella A, Rosen MJ, Winters J, Trepagnier C. Aspects of human factors engineering in home telemedicine and telerehabilitation systems. Telemed J. 1999;5:169–175. MEDLINE |
CrossRef
19. 19Kim J, Brienza DM. The virtual reality telerehabilitation system for analyzing accessibility of the physical environment: a comparison of camera systems. In: Proceedings of the RESNA 26th International Conference; 2003 June 19-23; Atlanta (GA). 20. 20Kim J, Brienza DM. The virtual reality telerehabilitation system for analyzing accessibility of the physical environment: a feasibility test. In: Proceedings of the RESNA 28th International Conference; 2005 June 25-27; Atlanta (GA). 21. 21Kim J, Brienza DM. Development of a virtual reality telerehabilitation system for analyzing accessibility of the built environment. J Rehabil Res Dev. 2006;43:257–272. MEDLINE |
CrossRef
22. 22Kim J. Development and effectiveness evaluation of a virtualized reality telerehabilitation system for accessibility analysis of built environment [dissertation]. Pittsburgh: Univ Pittsburgh; 2005;. 23. 23Kim J, Brienza DM. The virtual reality telerehabilitation system for analyzing accessibility of the physical environment: a field test. In: Proceedings of the RESNA 29th International Conference; 2006 June 22-26; Atlanta (GA). 24. 24Karlovits P. Home renovations make life easier for the physically challenged. http://www.pittsburghlive.com/x/pittsburghtrib/search/s_120925.html. 25. 25Uebersax JS. Structural analysis of diagnostic disagreements. J Nerv Ment Dis. 1983;171:199–206. MEDLINE |
CrossRef
a VA Center of Excellence in Wheelchairs and Associated Rehabilitation Engineering and the Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA b Departments of Rehabilitation Science and Technology, University of Pittsburgh, Pittsburgh, PA c Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA d Lynch & Associates, Architects, Pittsburgh, PA.  Correspondence to Jongbae Kim, PhD, c/o Christine Heiner, Human Engineering Research Laboratories, VA Pittsburgh Healthcare System (151R-1), 7180 Highland Dr, Bldg 4, 2nd Fl E, Pittsburgh, PA 15206
Supported by the Department of Veterans Affairs, Rehabilitation Research and Development Service (grant nos. B3142C, B2159T). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated. Reprints are not available from the author. PII: S0003-9993(07)01739-X doi:10.1016/j.apmr.2007.08.158 © 2008 American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation. Published by Elsevier Inc. All rights reserved. | |
|
FULL TEXT
