Physical Therapy Korea

In addition to the high incidence of dementia, Korea is currently recording the world’s highest rate of aging. According to data from the Statistics Korea, 16.6% of the total population was aged 64 or older in 2021. It is expected to jump to 20.6% in 2025 and 25.5% in 2030 and enter a super-aged society. About 15% of dementia is reversible dementia that can be recovered if detected early. In addition, even if it is irreversible dementia, early detection and therapeutic intervention can delay the progression of dementia symptoms [1]. However, most dementia patients in Korea visit hospitals only when their symptoms are severe. Therefore, patients with mild dementia should be found in the community [2].

As social problems arise due to the acceleration of the aging era and the increase in the elderly population, advanced information & communications technology (ICT) technology convergence healthcare is drawing attention. Currently, as ICT convergence healthcare is drawing attention, smart healthcare, such as disease prevention and early diagnosis using virtual reality (VR), augmented reality, and artificial intelligence (AI), is developing. Particularly, studies using VR as a therapeutic tool for cognitive impairment continue to emerge. Previously, VR was mainly used as a tool to assist exercise. Recently, the possibility of using it as a cognitive treatment tool has been expanding [3]. According to previous studies, VR-based cognitive training provides various senses and induces strong motivation in an interesting game format, improving cognitive function [4]. In addition, VR cognitive training is applied to mild cognitive impairment patients with new technologies, showing beneficial effects on memory, enforcement functions, and various cognitive domains [5].

In a study by Ju [6], it is stated that VR is worth using as a therapeutic medium for symptoms such as decreased motivation, absent mindfulness, and decreased movement in dementia patients. Cogn^é et al. [7]’s study reports that visual cues provided in VR help patients with Alzheimer’s dementia to execute and remember when performing a path-finding task within VR. Kim et al. [8] claimed that VR-based cognitive training could create a training environment for the elderly’s real-life activities, allowing them to experience and act with satisfaction in a space called VR, and that enhancing concentration through fun and interest can help prevent dementia and improve cognitive function. Zaj^ąc-Lamparska et al. [9] stated that game programs related to daily life, which include cognitive elements of memory, attention, language, and space-time, can improve the cognitive function during VR-based cognitive training, which can be an effective intervention for the elderly. Marquardt et al. [10] also reported that providing clues for work performance within the environment where dementia patients live can increase motivation for work performance and reduce errors in performance.

As such, various cognitive training using VR are continuously being studied. However, research on VR cognitive evaluation programs to prepare for various problems related to mild cognitive impairment is still insufficient. With the importance of mild cognitive impairment patient management system, this study attempted to reveal the possibility of clinical value and utilization through the validity and reliability analysis of the VR cognitive rehabilitation system.

1. Subjects

Elderly subjects were recruited from a senior welfare facility in Daejeon. The study was conducted on elderly people aged 65 to 85 with 24 points or higher on the Korean version of the Mini-Mental State Exam (K-MMSE; grades 1 to 3). The number of subjects was calculated using the G*Power (ver. 3.1.9.2; Heinrich Heine University D^üsseldorf) program, a sample count calculation program according to Cohen’s sampling formula (Table 1). Clinical trials were conducted on 26 people (12 in the normal control group and 14 in the mild cognitive impairment group) to verify the VR cognitive rehabilitation system. In addition, there was no resistance to using Head Mounted Display (HMD) devices in VR systems, and only the elderly who understood the purpose of the study and agreed in writing to participate in the study participated. This study was approved by the Daejeon University Ethics Committee (IRB no. 1040647-202205-HR-001-03).

Table 1 . General characteristic of the participants.

Variable	Value (N = 26)
Age (y)	74.77 ± 8.03
Sex (male/female)	13/13
Education level
Elementary school graduate	5
Middle school graduate	4
High school graduate	8
College graduate or higher	9
K-MMSE-2	27.8 ± 2.63
GDS
Grade 1a	12
Grades 2b and 3c	14

Values are presented as number only or mean ± standard deviation. K-MMSE-2, Korean version of the Mini-Mental State Examination, 2nd edition; GDS, Global Deterioration Scale. aNo cognitive decline, bvery mild cognitive decline, cmild cognitive decline..

Inclusion criteria were: (1) aged 65 to 85, (2) 24 points or higher on the K-MMSE (grades 1 to 3), (3) no objection to using HMD devices in VR systems, and (4) understood the purpose of the study and agreed in writing to participate in the study. Exclusion criteria were: (1) elderly people with difficulty understanding and using VR devices, (2) people complaining of physical side effects (e.g., headache, nausea, and dizziness) in using VR HMD devices through practice, and (3) people with a neurological disorder that makes it difficult to perform other cognitive tasks.

2. Experimental Procedures

The subjects individually conducted VR-based cognitive evaluation and existing paper-based cognitive evaluation (K-MMSE) at an independent place prepared in the elderly welfare center. In the basic evaluation, simple personal information, such as the names, ages, and gender of the study subjects, were collected. After that, the existing cognitive evaluation tool, the K-MMSE evaluation, was conducted, and the subjects were rated on the Global Deterioration Scale (GDS) through this. The VR cognitive rehabilitation system evaluation minimized the carryover effect by giving a 30-minute break between the test and retest. Each subject conducted the basic evaluation (5 minutes), K-MMSE evaluation (10 minutes), and VR cognitive rehabilitation system evaluation twice (20 minutes). It took about an hour for each subject, including a break.

Three studies were conducted with three evaluations conducted, and the research methods are as follows.

Criterion-related validity was investigated through correlation analysis between the K-MMSE measurement score and the VR cognitive evaluation system measurement score. For 26 subjects, the scores measured through the two cognitive evaluation tools were correlated.

The construct validity was investigated by comparing the two groups with the measurement scores of the VR cognitive evaluation system in the normal cognitive level group (GDS grade 1) and the very mild and mild cognitive impairment level groups (GDS grades 2 and 3).

First, participants were classified into two groups (GDS 1/GDS 2 and 3) through the K-MMSE evaluation tool. The t-test was performed with the evaluated VR cognitive evaluation score. Based on the t-test results, we analyzed the score difference between the normal group and the mild cognitive impairment group. Based on research by Choi et al. [11], the GDS grade was classified as no cognitive impairment if the K-MMSE score was 28.3 or higher (GDS grade 1), very mild cognitive impairment if it was 26.6 or higher (GDS grade 2), and mild cognitive impairment if it was 24 or higher (GDS grade 3).

An intra-rater reliability study in the form of test-retest reliability of the VR cognitive evaluation system was conducted. To evaluate the reliability of the VR cognitive evaluation system evaluation score, the first evaluation and the second evaluation (test-retest) were conducted on the same subject.

3. Instrumentations

We used the Korean version of the Mini-Mental State Examination version 2: standard version (K-MMSE-2:SV). The Korean version of the Mini-Mental State Examination, 2nd edition was standardized and published recently. The K-MMSE-2:SV used in this study consists of 11 questions (memory registration, time history, place history, memory recall, attention and calculation, language, and drawing) [12]. The total score is calculated as the sum of the scores of each question. Like the K-MMSE, the lowest score is 0, and the highest score is 30.

The VR cognitive evaluation system used in this study was prototype model (prototype, CMG Co., Ltd.) produced in cooperation with CMG. This system comprises 10 categories, consisting of two sub-series questions for five areas, and 10 questions, one from each category, were organized to be presented randomly. The five areas and the 10 sub-areas are shown in Table 2.

Table 2 . Developmental test of visual perception.

Category	Detail
VM	Object recognition, model recognition
FG	Object puzzle, model puzzle
PC	Object constancy, model constancy
PS	Object-oriented body instruction, subject-oriented body instruction
SR	Object-centered directional classification, subject-centered directional classification

VM, visual–motor coordination; FG, figure–ground perception; PC, perceptual constancy; PS, perception of position in space; SR, perception of spatial relationship..

Visual–motor coordination is the ability to coordinate vision with body movements. The test was conducted by drawing straight, curved, and bending lines along wide and narrow paths and connecting dots straight to determine eye–hand coordination ability. Figure–ground perception is the ability to clearly perceive the object of interest and attention. The test was conducted by perceiving and finding specific shapes in an increasingly complex background.

Perceptual constancy refers to the ability to recognize shapes despite changes in shape, position, or size. The test was conducted by recognizing the basic properties of a specific geometric shape among various shapes and determining the ability to discriminate differences from similar shapes.

Perception of position in space refers to the ability to perceive the front, back, top, and bottom by recognizing the relationship between the observer and the object, with the observer at the center. This test was conducted by discriminating between upside-down pictures and rotated pictures among a series of pictures presented in VR.

Perception of spatial relationship is the ability to identify and recognize an object and another object or the relationship between one object and another object. This test determined the ability to analyze simple shapes by looking at the presented shapes and connecting dots to match them (Table 3).

Table 3 . Example of VR evaluation system screen.

Category	Examples of question	Sample answer
VM
FG
PC
PS
SR

VR, virtual reality; VM, visual–motor coordination; FG, figure–ground perception; PC, perceptual constancy; PS, perception of position in space; SR, perception of spatial relationship..

Ten points were given for the correct answer for each question, and the weight was differentiated to 100%, 80%, and 60% compared to the reference time required to select the response. In other words, if the answer was selected and carried out within the normal reference time, 100 points were evaluated. If it exceeds the normal time, differential scores were given with 80, 60, and 0 points according to the set criteria. The total score of the 10 questions was summed up and derived as the final index score.

4. Data Analysis

All data were analyzed using IBM SPSS Window (ver. 22.0, IBM Co.). The general characteristics of the study subjects were technically analyzed, and a normality test was conducted using the Shapiro–Wilk test. Pearson correlation analysis was conducted to determine the validity of the reference criteria of the VR cognitive evaluation system. An independent t-test was conducted to determine the construct validity of the VR cognitive evaluation system. Finally, the reliability of the VR cognitive evaluation system was analyzed using the intraclass correlation coefficient (3,1).

1. Criterion-related Validity Analysis of the VR Cognitive Evaluation System

The R&D VR cognitive evaluation system showed a high correlation of 0.85 with the generally widely used K-MMSE cognitive evaluation. Therefore, it was statistically significant.

2. Construct Validity Analysis of the VR Cognitive Evaluation System

Compared to the average score of the VR cognitive evaluation system in the normal cognitive group (GDS grade 1), the average score of the VR cognitive evaluation system in the very mild and mild cognitive impairment level groups (GDS grades 2 and 3) was statistically low (Table 4).

Table 4 . Comparison of cognitive evaluation scores between the normal cognitive group (GDS grade 1) and very mild and mild cognitive impairment level groups (GDS grades 2 and 3).

Category	Normal cognitive groupa (n = 12)	Cognitive impairment group [very mildb-mildc] (n = 14)	t	p-value
VR cognitive evaluation (scores)	88.33 ± 12.23	61.43 ± 14.24	5.120	0.001
K-MMSE (scores)	29.08 ± 0.90	25.71 ± 1.06	8.605	0.001

Values are presented as mean ± standard deviation. GDS, Global Deterioration Scale; VR, virtual reality; K-MMSE, Korean version of the Mini-Mental State Exam. aGDS grade 1, bGDS grade 2, cGDS grade 3..

3. Reliability of the VR Cognitive Evaluation System

The analyzed intraclass correlation coefficient was 0.923. It was statistically significant (α < 0.05). It also showed excellent reliability at a 95% confidence level (Table 5).

Table 5 . Reliability analysis of the VR cognitive evaluation system.

Intraclass correlation coefficient	95% confidence interval			F test with true value 0
	Lower bound	Upper bound		Value	df1	df2	Significance
0.923	0.836	0.965		25.042	25	25	< 0.05

VR, virtual reality..

This study attempted to reveal the clinical value of the VR cognitive evaluation system by analyzing the validity and reliability of the VR cognitive evaluation system. As a result, it showed a high correlation (0.85) with the K-MMSE, an evaluation tool useful in diagnosing dementia. In other words, it can be said that it shows a similar pattern of 85% compared to the evaluation system that is already widely used. Since it is highly similar to the K-MMSE, the VR cognitive evaluation system developed in this study has the validity to be used clinically. In addition, in the analysis of the construct validity of the VR cognitive evaluation system, the VR cognitive evaluation system proved the construct validity that can distinguish between normal cognitive status (GDS grade 1) and mild cognitive impairment (GDS grades 2 and 3). This shows the possibility of VR cognitive evaluation systems being used clinically. In addition, as a result of analyzing the reliability of the VR cognitive evaluation system, this system was identified as a stable and reliable system capable of continuous evaluation with consistent scores.

Early detection of mild cognitive impairment is important because appropriate action at the mild cognitive stage can inhibit or slow progression to dementia [13]. Jeong [14] stated that diagnosing mild cognitive impairment and detecting Alzheimer’s at the earliest can maximize the therapeutic effect. However, while there are currently many studies that prove the effectiveness of VR as a therapeutic tool, studies using VR as a cognitive function evaluation tool are still lacking. Therefore, this study is meaningful because it used its own VR program as an evaluation tool and proved its validity and reliability.

Before starting this study, the concern was that the subjects were the elderly who had difficulty accessing the evaluation tool. When conducting the experiment on the elderly, there was no major difficulty in the study’s progress because we conducted it on the elderly with normal to mild cognitive impairment levels. However, it is unclear whether the VR program used in this study can be applied to the elderly with severe dementia. In addition, many elderly people are not familiar with VR yet, so it may be difficult to apply it to real life right away.

Furthermore, the side effects of VR must be considered. Previous studies that used VR as a rehabilitation treatment tool claim that there are elderly people who cannot adapt to VR and who show symptoms such as motion sickness. In a study by Ju [6], it was reported that most rehabilitation treatments using VR were shorter than traditional rehabilitation treatments because providing VR for a long period could cause side effects such as dizziness and vomiting. In our study, the cognitive evaluation using VR was conducted for about 20 minutes, so there was no rejection from the subjects in the experiment. However, one should be cautious of using VR for long periods because there is a risk of accumulating cognitive fatigue.

At the time of the test–re-test reliability evaluation, we added a 30-minute break between tests to minimize the carryover effect and control the subject’s condition. However, this does not seem to have completely prevented the carryover effect. It is believed that the short break time affected the reliability test within the tester. Kim et al. [15] attempted a second measurement at least 24 hours after the first measurement when evaluating reliability. In the future, it would be useful to supplement this method in a study that verifies the reliability of the cognitive evaluation tool using VR. With the rapid rise of smart health care, exercise rehabilitation and cognitive rehabilitation using VR have continued to develop. However, while there have been many studies proving the effectiveness of VR as a treatment tool, studies using VR as a cognitive evaluation tool are still lacking. Therefore, this study is meaningful because it used VR as an evaluation tool and proved its validity and reliability.

The limitations of this study are as follows. The target number of subjects for this study, which we calculated using G*power ver. 3.1.9.2, was 30. However, due to the difficulty in recruiting subjects because of the COVID-19 pandemic and the fact that the elderly with a lower GDS grade than grade 3 were eliminated, the study was conducted with 26 people, four less than the target number.

In addition, since the subjects of this study were elderly people with GDS grades 1 to 3, the VR cognitive evaluation system developed for this study is difficult to prove for severe dementia patients. However, in a study that examined the cognitive function of stroke patients using a 3D VR program by Kim et al. [15], there was a strong correlation between the K-MMSE and VR cognitive function evaluation programs, and there was a significant difference in comparison between groups according to the degree of cognitive function. Therefore, it can compensate for the limitations of our study.

To compensate for these limitations, cognitive evaluation programs using reliable VR that can be used for mild to severe dementia should continue to be developed in the future. Furthermore, it is hoped that there will be many VR cognitive evaluation programs that can be used comfortably by the public to prevent dementia.

This study sought to determine the clinical utility of the R&D VR cognitive rehabilitation system through validity and reliability analyses. The VR cognitive evaluation system was confirmed as an evaluation tool that can be used clinically with validity and reliability to distinguish between normal cognitive status and mild cognitive impairment.

None.

None to declare.

No potential conflicts of interest relevant to this article are reported.

Conceptualization: JC. Data curation: JK, SK. Formal analysis: JK, SK. Investigation: JK, SK. Methodology: JK, SK. Project administration: JC. Supervision: JC. Validation: SK. Writing - original draft: JK, SK. Writing - review & editing: JC.