Physical Therapy Korea
eISSN 2287-982X
QR Code
eISSN 2287-982X
Article
Article Tools
Share this article on :
Starts or metrics
Article
Original Article
Split Viewer
Phys. Ther. Korea 2023; 30(4): 306-313
Published online November 20, 2023
https://doi.org/10.12674/ptk.2023.30.4.306
© Korean Research Society of Physical Therapy
Verification of the Reliability and Validity of a Virtual Reality Cognitive Evaluation System Based on Motion Recognition Analysis Evaluation
Jeonghan Kwon1 
, PT, BPT, Subeen Kim2 , PT, BPT, Jongduk Choi3 , PT, PhD
1Rehabilitation Center, Catholic Kwandong University International St. Mary’s Hospital, Incheon, 2Rehabilitation Department, Daejeon Public Children Rehabilitation Hospital, 3Department of Physical Therapy, College of Health and Medical Science, Daejeon University, Daejeon, Korea
Correspondence to: Jongduk Choi
E-mail: choidew@dju.kr
https://orcid.org/0000-0002-9663-4790
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background: As social problems due to the acceleration of the aging era and the increase in the elderly population are becoming serious, virtual reality (VR)-based healthcare is emerging as an approach for preventing and managing health issues. Objects: This study used validity and reliability analyses to examine the clinical efficacy that is, the clinical value and usability of a novel VR cognitive evaluation system index that we developed.
Methods: We developed a VR cognitive evaluation system based on motion recognition analysis evaluation for individuals aged 65 to 85. After conducting the Korean version of the Mini-Mental State Exam (K-MMSE) cognitive evaluation, the evaluation score was verified through correlation analysis in the VR cognitive evaluation system. To verify the construct validity of the two groups, the Global Deterioration Scale (GDS) grades were categorized into a normal cognitive group (GDS grade 1) and a cognitive impairment group (GDS grades 2 and 3). The data were measured twice to determine the reliability between the two measurements and assess the stability and clinical value of the evaluation system.
Results: Our evaluation system had a high correlation of 0.85 with the widely used K-MMSE cognitive evaluation. The system had strong criterion-related validity at the 95% confidence interval. Compared to the average score of GDS grade 1 in the VR cognitive evaluation system, the average score of GDS grades 2 and 3 in the VR cognitive evaluation system was statistically significantly lower while also having strong construct validity at the 95% confidence interval. To measure the reliability of the VR cognitive evaluation system, tests–retests were conducted using the intraclass correlation coefficient (3,1), which equaled 0.923 and was statistically significant.
Conclusion: The VR cognitive evaluation system we developed is a valid and reliable clinical tool to distinguish between normal cognitive status and mild cognitive impairment.
Keywords: Cognitive dysfunction, Risk assessment, Virtual reality
PDF
Standard view
Export citation
Share Article
Original Article
Phys. Ther. Korea 2023; 30(4): 306-313
Published online November 20, 2023 https://doi.org/10.12674/ptk.2023.30.4.306
Copyright © Korean Research Society of Physical Therapy.
Verification of the Reliability and Validity of a Virtual Reality Cognitive Evaluation System Based on Motion Recognition Analysis Evaluation
Jeonghan Kwon1 , PT, BPT, Subeen Kim2 , PT, BPT, Jongduk Choi3 , PT, PhD
1Rehabilitation Center, Catholic Kwandong University International St. Mary’s Hospital, Incheon, 2Rehabilitation Department, Daejeon Public Children Rehabilitation Hospital, 3Department of Physical Therapy, College of Health and Medical Science, Daejeon University, Daejeon, Korea
Correspondence to:Jongduk Choi
E-mail: choidew@dju.kr
https://orcid.org/0000-0002-9663-4790
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Background: As social problems due to the acceleration of the aging era and the increase in the elderly population are becoming serious, virtual reality (VR)-based healthcare is emerging as an approach for preventing and managing health issues. Objects: This study used validity and reliability analyses to examine the clinical efficacy that is, the clinical value and usability of a novel VR cognitive evaluation system index that we developed.
Methods: We developed a VR cognitive evaluation system based on motion recognition analysis evaluation for individuals aged 65 to 85. After conducting the Korean version of the Mini-Mental State Exam (K-MMSE) cognitive evaluation, the evaluation score was verified through correlation analysis in the VR cognitive evaluation system. To verify the construct validity of the two groups, the Global Deterioration Scale (GDS) grades were categorized into a normal cognitive group (GDS grade 1) and a cognitive impairment group (GDS grades 2 and 3). The data were measured twice to determine the reliability between the two measurements and assess the stability and clinical value of the evaluation system.
Results: Our evaluation system had a high correlation of 0.85 with the widely used K-MMSE cognitive evaluation. The system had strong criterion-related validity at the 95% confidence interval. Compared to the average score of GDS grade 1 in the VR cognitive evaluation system, the average score of GDS grades 2 and 3 in the VR cognitive evaluation system was statistically significantly lower while also having strong construct validity at the 95% confidence interval. To measure the reliability of the VR cognitive evaluation system, tests–retests were conducted using the intraclass correlation coefficient (3,1), which equaled 0.923 and was statistically significant.
Conclusion: The VR cognitive evaluation system we developed is a valid and reliable clinical tool to distinguish between normal cognitive status and mild cognitive impairment.
Keywords: Cognitive dysfunction, Risk assessment, Virtual reality
-
Table 1 . General characteristic of the participants.
Variable Value (N = 26) Age (y) 74.77 ± 8.03 Sex (male/female) 13/13 Education level Elementary school graduate 5 Middle school graduate 4 High school graduate 8 College graduate or higher 9 K-MMSE-2 27.8 ± 2.63 GDS Grade 1a 12 Grades 2b and 3c 14 Values are presented as number only or mean ± standard deviation. K-MMSE-2, Korean version of the Mini-Mental State Examination, 2nd edition; GDS, Global Deterioration Scale. aNo cognitive decline, bvery mild cognitive decline, cmild cognitive decline..
-
Table 2 . Developmental test of visual perception.
Category Detail VM Object recognition, model recognition FG Object puzzle, model puzzle PC Object constancy, model constancy PS Object-oriented body instruction, subject-oriented body instruction SR Object-centered directional classification, subject-centered directional classification VM, visual–motor coordination; FG, figure–ground perception; PC, perceptual constancy; PS, perception of position in space; SR, perception of spatial relationship..
-
Table 3 . Example of VR evaluation system screen.
Category Examples of question Sample answer VM 
FG 
PC 
PS 
SR 
VR, virtual reality; VM, visual–motor coordination; FG, figure–ground perception; PC, perceptual constancy; PS, perception of position in space; SR, perception of spatial relationship..
-
Table 4 . Comparison of cognitive evaluation scores between the normal cognitive group (GDS grade 1) and very mild and mild cognitive impairment level groups (GDS grades 2 and 3).
Category Normal cognitive groupa
(n = 12)Cognitive impairment group
[very mildb-mildc] (n = 14)t p-value VR cognitive evaluation (scores) 88.33 ± 12.23 61.43 ± 14.24 5.120 0.001 K-MMSE (scores) 29.08 ± 0.90 25.71 ± 1.06 8.605 0.001 Values are presented as mean ± standard deviation. GDS, Global Deterioration Scale; VR, virtual reality; K-MMSE, Korean version of the Mini-Mental State Exam. aGDS grade 1, bGDS grade 2, cGDS grade 3..
-
Table 5 . Reliability analysis of the VR cognitive evaluation system.
Intraclass correlation
coefficient95% confidence interval F test with true value 0 Lower bound Upper bound Value df1 df2 Significance 0.923 0.836 0.965 25.042 25 25 < 0.05 VR, virtual reality..