Virtual Harmony: Music interaction with virtual reality to reduce stress

This paper presents a demonstration of a newly created device called Virtual Harmony, designed to address and reduce stress by applying Virtual Reality (VR) in a Music Therapy (MT) environment. The treatment combines VR and MT to stimulate audition through background music, vision through three-dimensional VR, and touch through virtual percussive instruments. A pilot study was conducted among 19 high school and college students, in which participants used Virtual Harmony and provided pre- and post-exposure questionnaire responses about their experiences. 90% of participants reported that Virtual Harmony was effective and worth purchasing and 32% of participants reported a significant decrease in stress levels after having used the device. Only 5% reported that their stress increased which may be related to past experiences with severe vertigo. Further controlled experimentation is needed, but these early results are consistent with Virtual Harmony being a promising, affordable, and accessible way for users to manage their stress.


Introduction
Stress is a common experience in daily life for most people. 75% of adults reported feeling stressed in the previous month (APA 2017). Despite the ubiquity of stress, not all people know how to mitigate it. Unfortunately, in many instances, the professional treatments for reducing stress are expensive and inaccessible. One of the most affected populations of stress is college students, reaching as high as 89% feeling stressed 2 to 4 times per semester (MentalHelp 2016). Not only do Millenials report the highest levels of stress, they also seem to rely on sedentary stress management techniques -listening to music being the most common (APA 2015). In addition to the financial struggle of affording therapies, students also face stigmas against psychological services, a lack of transportation, and a limited amount of time.
This study explores a new application developed by the authors, Virtual Harmony, intended to supplement treatment options for stress. The application addresses obstacles to treatment by providing an accessible and affordable platform to reduce the effects of stress through the combination of Virtual Reality technology and Music Therapy practices. Moreover, like Music Therapy, the device can be used by musicians and nonmusicians alike. Figure 1 illustrates a user engaged with Virtual Harmony. 1

Related Devices
Other developers have created similar devices to try to mitigate stress for their users. Mitrpanont et. al. (2017) created the iCare-Stress, a device that processes brainwaves to determine the user's stress levels, which then suggests music therapy practices based on the results. Pingle (2016) developed an Internet of Things (IOT) that recommends music through a web interface after collecting biodata. Mahalil et. al (2014) created a stress relieving therapy that combines VR and music therapy-based visualization using forest scenes coupled with audio elements such as a waterfall and animals. Similarly, Thoondee and Oikonomou (2017) also proposed a VR based system to reduce stress and provoke relaxation at work during breaks. Jiang et. al. (2013) designed the Intelligent Move Therapy System for Mental Disease Treatment to calm the stress and emotions of users through a movie, music, and a plot.
In contrast to these many devices, the proposed Virtual Harmony device provides both active and receptive music therapy in a peaceful VR threedimensional environment with soothing music. Virtual Harmony's most novel feature is its combination of several therapeutic techniques. Further, studies reveal that listening to music is the most common form of coping with stress, particularly among Millenials (APA 2015). While listening to music and watching videos is certainly a common destressing tool, Virtual Harmony provides yet another sensual interaction through touch.

Music Therapy
In recent years, music therapy has become the most commonly prescribed treatment for complementary and alternative medicine (CAM), as it affects both the central and peripheral nervous systems (Ferrer 2014).
One benefit of music in this kind of application is that it can manipulate time through beat, rhythm, and pulse. To encourage time-based manipulations, the Virtual Harmony prototype offers virtual percussive instrument selections, namely piano, xylophone, and a drum set, as shown in Figure 2. Furthermore, the auditory cortex, located deep within the cerebrum, is very close to the limbic system and hippocampus where the brain stores memories. Thus, the auditory cortex may be able to retrain neural and behavioral functions to maintain relief and comfort even outside of the therapeutic environment (Ferrer 2014).
The decision was made to include background music to simulate the experience of music therapy, in which the patient musically interacts with the music production of the therapist. Specifically, the Virtual Harmony prototype uses the second movement of Beethoven's 7th Symphony. There are several reasons why this musical selection is appropriate to simulate a music therapy session. First, the slow tempo of 72 beats per minute is similar to the average heart rate, so elevated heart rates due to stress may slow down to match the tempo, conducive for reducing stress. Second, as with several of Beethoven's slow movements, the melody cycles in about ten seconds -the amount of time it takes a signal to travel from the brain to the heart to regulate a healthy blood pressure (Young 2015). Lastly, less raucous and more tranquil music stimulates the brain, triggering chemicals that suppress the production of adrenaline and corticosteroids, hormones directly related to high levels of arousal and stress (Dove 2009).

Figure 2: Examples of the interface of Virtual Harmony with the three instrument options.
Virtual Reality Virtual Reality (VR) is a newly refined technology which immerses the user in a transformed, present threedimensional environment. Not only does VR provide an accessible and affordable vacation from collegiate chaos, recently it has also emerged as a therapeutic device, specifically as an antidepressant (Lindner et. al. 2019). Furthermore, in a study conducted comparing 3D videos and 2D videos, participants reported feeling more distracted when watching a 3D video (Nanda 2011).

Cognitive Load Theory
The most important feature of Virtual Harmony is the virtual percussive instruments (see Figure 2), because it provides yet another stimulation to distract the brain. Educational psychologist, John Sweller, (1988) developed the cognitive load theory which refined the understanding that sensory register memory can only intake a limited number of perceptions at once. Therefore, by distracting three of the five senses, Virtual Harmony creates a virtually real environment that medializes the participant before reaching the threshold of being overwhelmed.

Cognitive Behavioral Therapy
Virtual Harmony follows Cognitive Behavioral Therapy practices which focus on changing unhelpful cognitive distortions and behaviors, improving emotional regulation, and developing personal coping methods (Otte 2011). First of all, unlike a traditional music therapy setting, the environment is customizable. Secondly, the system operates in both receptive and active modes. In the receptive mode of operation, users experience the 360° panoramic video and background music. Once accustomed to the new interface, users can participate in the active mode of operation with virtual percussive instruments. In this way, users can "communicate" with the background music, much like a typical music therapy setting.

Technology Design
Virtual Harmony uses two types of interfaces: the headmounted display (HMD) and the hand-tracker. The HMD, or the headset, is used to engage the user fully in the virtual experience. Once wearing the headset and in the virtual world, users cannot see anything in the real world. Furthermore, through the HMD, users can look around the environment which gives them control of the scene and feedback for motion. See Figure 3 for examples of the two environments used in the prototype. After putting on the headset, users will see their virtual hands through the Leap Motion hand-tracker. This innovative controller-less system replaces the traditional joystick or keyboard to give the user a more realistic sensation (see Figure 4). After putting on the headset, users will see two virtual hands and the Leap Motion captures the movements of their real hands. Users are able to move their hands as well as each finger freely. When the left hand is facing palm up, a menu of background music, 3D videos, and instruments will open. Using the right hand, users can select a background piece, a video, and as many instruments as they prefer. Once in the virtual world, users can interact with background music as they wish with the instruments provided, or they can turn to observe what is happening visually in their surroundings. Even if the background music or video ends, they automatically repeat.
While not a component of the final prototype used in the study, we prerecorded a voice that guides users through navigating the virtual world as well as suggesting meditative relaxing techniques such as breathing deeply and closing the eyes.

Hardware
The Virtual Harmony prototype was developed with a VR Ready gaming laptop, specifically Intel(R) Core(TM) i7-7820HK CPU @ 2.90GHz (8 CPUs), 32768MB RAM, NVIDIA GeForce GTX 1080 and Windows 10. In addition, it also uses a professional video game engine called Unity for the software development platform. The head mount display device is the consumer grade VR headset, HP Windows Mixed Reality Headset, with 1440 x 1400 resolution per eye and 90 HZ refresh rate. Leap Motion, a world-leading hardware used for the hand tracking system, was attached to the front of the headset. The current marketing prices for these two devices are $179 and $89 respectively. However, while we used high performance systems, Virtual Harmony can still function under less powerful machines such as the Oculus Go, currently $149.

Method
Due to the circumstances of the COVID-19 pandemic, thorough human participant research was not possible. Nevertheless, we conducted a pilot study of 19 volunteer participants, all high school and college students. Each participant was asked to complete a pre-survey regarding their stress level and general information such as age, education, level, and previous experiences with vertigo. After demonstrating how to use the device, we encouraged users to explore the virtual world presented. While we gave no time limit, participants spent an average of 11.2 minutes inside the simulation. Afterwards, participants filled out a post-survey regarding their experience and if their original level of stress was affected.

Pre-Survey
The pre-survey asked participants a range of background questions, including age, gender, and education level. It then asked several generalized questions about current stress levels, sources of stress, and strategies the participant used for managing stress. Specifically, we asked participants how they usually coped with stress from a list of options: music (listening or playing), reading, sports/exercise, eating, sleeping, socializing with friends, I don't feel as if I have any coping, and other. We also asked whether participants had visited a therapist, and if not, why not.

Post-Survey
Once the study was completed, participants were asked a number of follow-up questions about their experience with Virtual Harmony. On a scale of 1-10, they were asked to rate their experience with Virtual Harmony and their stress level after using it. They were then asked whether they would prefer using Virtual Harmony over seeing a therapist in person. Then they were asked whether they had any negative physical reactions to the application, including general discomfort, fatigue, headache, nausea, difficulty concentrating, or dizziness. Finally, they were asked how much they would be willing to pay for the system.

Results
The participants in the pilot study reported that music is indeed their most preferred coping method for dealing with stress (see Figure 5). Further, as shown in Figure 6, a majority of the participants reported that their overall stress levels significantly decreased after using Virtual Harmony (p < .0001). Specifically, on a scale of 0-10, mean participant stress decreased from a pre-test mean of 6.7 to a post-test mean of 3.5.
Notably, just one participant's stress level increased after using the device. However, this participant was also the only one to have reported suffering from severe vertigo, epilepsy, or a similar condition. Wearing the headset caused four additional participants to feel slight motion sickness. Even still, all five users were interested in participating in testing the next version of our system in the future. Figure 6: Self-reported post-test stress levels (m = 3.5) significantly lower than pre-test stress levels (m = 6.7), p < .0001.

Discussion
The preliminary experimental results of Virtual Harmony are consistent with a positive effect of combining the two therapeutic techniques of Virtual Reality and Music Therapy to reduce participant stress levels. Some participants experienced motion sickness using Virtual Harmony, and of these participants, one experienced increased stress. While the occasions of motion sickness can be disheartening, as with any treatment, negative side effects can occur for some patients. Some participants may enjoy VR as a supplement to traditional therapies, but this technology may not be for all stress patients. Further, elements of the technology may have been cause for distress, especially the virtual piano, as the virtual keys are difficult to coordinate. This negative experience is likely exacerbated for those without prior musical experience.

Conclusion
While Virtual Harmony is still in a prototypical stage, preliminary results promisingly affirm the therapeutic benefits demonstrated previously in Virtual Reality and Music Therapy practices. In future research, we hope to more thoroughly test the effectiveness of this treatment on stress through bio measures in comparison to traditional methods.

Future Goals
In addition to more formal experimentation, we have several visions for how we can expand user experience and improve the technology and technique. To begin with, the user interface (UI) design can be improved to better reflect human behavior. We will also consider increasing the size of the graphical user interface (GUI), so that users can understand and control the environment more simply. Secondly, we hope to develop a platform where users can personally interact with a professional music therapist through the virtual environment. Lastly, we want to design an application that allows users to select individualized music, 360 o videos, and virtual instruments. We would also offer packages of these three components as recommended by music therapists for purchase. Otherwise, users can simply upload their custom interests. Ultimately, we hope to one day implement Virtual Harmony on college campuses to reduce stress in a way that is affordable, accessible, and effective.