This experimental protocol demonstrates the use of virtual prism adaptation therapy (VPAT) in healthy adults and the association between VPAT and functional near infrared spectroscopy to determine the effect of VPAT on cortical activation. Results suggest that VPAT may be feasible and could induce similar behavioral adaptation as conventional prism adaptation therapy.
Hemispatial neglect is a common impairment after stroke. It is associated with poor functional and social outcomes. Therefore, an adequate intervention is imperative for the successful management of hemispatial neglect. However, the clinical use of various interventions is limited in real clinical practice. Prism adaptation therapy is one of the most evidence-based rehabilitation modalities to treat hemispatial neglect. To overcome any possible shortcoming that may occur with prism therapy, we developed a new system using immersive virtual reality and depth-sensing camera to create a virtual prism adaptation therapy (VPAT). To validate the VPAT system, we designed an experimental protocol investigating the behavioral errors and changes in cortical activation via the VPAT system. Cortical activation was measured by functional near infrared spectroscopy (fNIRS). The experiment consisted of four phases. All four included clicking, pointing or rest applied to right-handed healthy people. Clicking versus pointing was used for investigating the cortical region related with the gross motor task, and pointing with VPAT versus pointing without VPAT was used for investigating the cortical region associated with visuospatial perception. The preliminary results from four healthy participants showed that pointing errors by the VPAT system was similar to the conventional prism adaptation therapy. Further analysis with more participants and fNIRS data, as well as a study in patients with stroke may be required.
Hemispatial neglect, which affects the ability to perceive the contralateral hemispatial visual field, is a common impairment after stroke1,2. Although rehabilitation after hemispatial neglect is important, due to its association with poor functional and social outcomes, rehabilitation is often underutilized in real clinical practice3,4.
Among the various existing rehabilitation approaches suggested for hemispatial neglect, prism adaptation (PA) therapy has proven effective for recovery and improvement in hemispatial neglect in patients with subacute or chronic stroke5,6,7,8. However, conventional PA is underutilized due to several drawbacks9,10. These include 1) high cost and time requirement due to the prism lens needing to be changed to adjust to the degree of deviation; 2) the need to set up additional materials to be pointed at and to mask the hand trajectory; and 3) PA can only be used by patients who can sit and control their head position.
A recent study reproducing the adaptation effects in the virtual reality (VR) environment reported that it may be possible for the virtual prism adaptation therapy (VPAT) to have different effects depending on the subtypes of neglect11. It was also suggested that cortical activation for PA may vary according to brain lesions12. However, little is known about the cortical activation pattern seen in VR-induced PA.
To overcome these obstacles and promote the use of PA in a clinical setting, we developed a new PA therapy system using an immersive VR technology called virtual prism adaptation therapy (VPAT), via the use of a depth-sensing camera. We designed an immersive VR system with the ability to provide visual feedback about the position of a virtual limb to promote spatial realignment13. Using this immersive VR technology, which mimicked the effect of conventional PA, we designed an experiment to validate the VPAT system in healthy participants.
By conducting our visualized experimental protocol, we investigated whether the new VPAT system can induce behavioral adaptation, similar to conventional PA. Additionally, we would like to explore whether the VPAT system can induce the activation in the cortical regions associated with visuospatial perception or recovery of hemispatial neglect after stroke.
All procedures were reviewed and approved by the Seoul National University Bundang Hospital Institutional Review Board (IRB). To recruit healthy participants, posters were used to advertise around the hospital.
1. Experimental set-up
2. Experimental set-up (Figure 2)
3. Experiment to validate VPAT system
4. Data analysis
Data from four healthy participants (1 man and 3 women) were used as representative results. A pointing error is shown in Figure 5A, with the averages of median value of 10 trials in the sub-phase of each pointing task lasting 30 s. Values on average for the median pointing errors in the first block of each phase were 0.45 ± 0.92 (pre-VPAT), 4.69 ± 3.08 (VPAT 10°), 5.43 ± 2.22 (VPAT 20°), and -5.17 ± 1.60 (post-VPAT). The trend of pointing error change was statistically significant (p = 0.001) via the repeated measures ANOVA. A pointing error for each subject is presented in Figure 5B, illustrating the adaptation during the VPAT phase and post-prismatic adaptation (negative pointing error).
Figure 1: Experimental setting with VPAT and fNIRS linkage system. VPAT = virtual prism adaptation therapy; fNIRS = functional near infrared spectroscopy. This figure was previously published by Kim et al.15 Please click here to view a larger version of this figure.
Figure 2: The subject performing the experiment with VPAT and fNIRS system. VPAT = virtual prism adaptation therapy; fNIRS = functional near infrared spectroscopy. Please click here to view a larger version of this figure.
Figure 3: Montage containing 54 channels by arranging 15 light sources (red circles) and 24 detectors (blue circles) at intervals of 3 cm. Space between the nearest sources and the detector constituted one channel, which is represented as yellow circles with a number. Please click here to view a larger version of this figure.
Figure 4: Experimental design. VPAT = virtual prism adaptation therapy; Pt = pointing; Cl = clicking; Re = resting. Please click here to view a larger version of this figure.
Figure 5: Pointing errors in each block. (A) Average value graph of subject's median pointing error in each block. This figure was previously published by Kim et al.15 (B) Median pointing error in each block by each subject. The counterclockwise direction (i.e. left from the target) is the positive value. Please click here to view a larger version of this figure.
This study implemented the prism adaption therapy using a translated hand movement in a VR environment. It investigated whether the deviation implemented was causing angle overshooting and behavioral adaptation, as in conventional prism adaption therapy.
In the median pointing error result (Figure 5) and the first pointing error result, the pointing error changed significantly when the phase was switched. Although some hand-recognition errors were eliminated, there may still be false detection. The use of a median value to eliminate systematic error, such as false tracking, showed that the average pointing error results were lower than expected. Post-prismatic adaptation was constantly shown in each subject (Figure 5B). These results showed similar behavioral adaptation to the conventional prism adaptation therapy.
There were some problems in the experiment. False detection of the hand occurred frequently in the pointing task. In some cases, even though the hand reached the target during pointing, the virtual hand was not tracked due to a Leap motion recognition error. In addition, because the participants were wearing HMD in the clicking task, it was difficult for them to locate the push button and the experimenter had to provide continuous assistance. The weight of the HMD and its long-term application could also cause pain in the area that comes into contact with the fNIRS optode. Therefore, there were times when the HMD was lifted or the participants themselves were holding the HMD.
If we overcome the shortcomings of the system and consolidate the results of the experiment through more data analysis, including fNIRS data, it could potentially be used in the treatment of visuospatial neglect. In addition, game-friendly contents can be applied to present an immersive and fun treatment modality. Nonetheless, further study with a more advanced VPAT system proving clinical efficacy in stroke patients with visuospatial neglect is needed.
Several previous studies have reported motion sickness induced by the use of Immersive VR, or head-mounted VR sets16. Motion sickness is reported to be infrequent if VR is implemented in seated positions17. Motion mismatch can also cause motion sickness, but it can be reduced by independently configuring the background in the virtual environment18,19. In this system, only the hand deviation angle caused motion mismatch, which should have less impact on motion sickness overall.
Participants in this experiment were normal adults, so there were no consistent problems. However, to be used as therapeutic treatment for stroke patients, the above issues need to be considered, and virtual prism therapy protocols need to be taken into account, such as taking breaks during treatment or the length of treatment time.
The authors have nothing to disclose.
This study was supported by the Seoul National University Bundang Hospital Research Fund (14-2015-022) and by the Ministry of Trade Industry & Energy(MOTIE, Korea), Ministry of Science & ICT(MSIT, Korea), and Ministry of Health & Welfare(MOHW, Korea) under Technology Development Program for AI-Bio-Robot-Medicine Convergence (20001650). We would like to thank Su-Bin Park, Nu-Ri Kim and Ye-Lin Jang for helping to prepare and proceed with the video shooting.
EASYCAP | Easycap | C-SAMS | Platform to accommodate fNIRS optodes |
Leap Motion 3D Motion Controller | Ultrahaptics | FBA_LM-C01-US | Hand detection device attached HMD |
Leap Motion VR Developer Mount for VR Headset | Ultrahaptics | VR-UAZ | |
Matlab R2015a | Mathworks | Programming language running with NIRStar | |
NIRScout | Medical Technology LLC | NSC-CORE | fNIRS system |
nirsLAB v201605 | Medical Technology LLC | Software for analyzing data collected with NIRScout | |
NIRStar 14.1 | Medical Technology LLC | NIRScout Acquisition Software | |
Occulus Rift DK2 | Occulus | VR HMD | |
PowerMate USB Multimedia Controller | Griffin Technology | NA16029 | Push Button in task |
SuperLab 5.0 | Cedrus Corp. | Synchronize the stimulus presentations allied to NIRScout |