Biosignals in VR Jazz Bar
Master Graduation Thesis
Designing and Evaluating Avatar Biosignal Visualization Techniques in Social Virtual Reality
DURATION
Feb 2021 - Aug 2021
(Individual Project)
TOOL
Miro, Abode Suite,
Unity (C#), Blender, HTC Vive
SKILL
User Research, Methodology Design, VR Design & Development
BACKGROUND
This project was a Design for Interaction Master's graduation thesis: "Designing and evaluating avatar biosignal visualization techniques in the social virtual reality spaces." The general theme was brought up by Centrum Wiskunde & Informatica (CWI): Sharing avatar biosignals in the social virtual reality spaces.
source - facebook
UNDERSTANDING THE CONTEXT
Biosignals
Biosignals are all kinds of signals that can be measured and monitored by biological beings.
e.g. beating heart, breathing, etc
Social Virtual Reality (Social VR)
Social VR is a virtual reality application that supports remote, multidimensional interaction in virtual space. (e.g. VRChat, etc)
In Social VR platforms, users interact with others through avatars, which are the human representation in VR spaces.
PROBLEM
The current avatar doesn’t reflect the user’s real physiological (biological) state. The gap often results in miscommunication between the users and degrades the immersion in the VR spaces.
GOAL
The goal is to visualize the human’s biosignals on the avatars in the Social VR so that the current avatars can better reflect the real state of the users and enhance the overall user experience.
RESEARCH PROCESS
These are the approaches taken to answer the research questions:
1. Requirements
This section answers the research questions and looks for the design opportunities in the social VR spaces. Insights from two design methodologies - survey and interview - will lead to setting the design direction of this project.
1.1 SURVEY
A survey was conducted to narrow down the scope of the research in terms of the target user, platform, and their purposes for using social VR platforms; the secondary goal was to find the potential candidates for the subsequent qualitative design process.
1.2 INTERVIEW
The followed by interviews - with six social VR users - were to explore the current experience of using avatars in different scenarios, roles, and environments in the social VR context.
A 'sensitizing booklet' - 3 days of activities that the interviewees had to fill in before the main interview session - was designed to help them reflect back on their social VR experience thereby being able to give more rich and in-depth responses.
1.3 RESULTS AND FINDINGS
[A] Current User Behavior & Interaction in Social VR
The type of platform or environment was a more influential factor compared to the type of relationship when people decorated their avatars; thus, one particular scene had to be chosen before starting to visualize biosignals.
[answer to RQ 1-1]
[B] Sharing Biofeedback in Social VR
People will be sharing biosignals only if the case satisfies one of the four objectives presented below.
[answer to RQ 1-2]
1.4 DESIGN DIRECTION
The objectives of sharing biosignals in social VR spaces were used as a stepping stone to compare different scenarios and choose a design direction. Additionally considering the practical limitations, the design direction was set to focus on 'Between audience interaction in watching entertainment scenario'.
2. Conceptualization
This section answers the question: "How to visualize avatar biosignals for immersive VR experience in watching entertainment scenarios?” The co-design session was conducted to concretize the design concepts.
2.1 CO-DESIGN SESSION
The co-design session was conducted with six designers to:
-
look for a suitable case of sharing biosignals within the watching entertainment scenarios;
-
collect biosignal visualizing ideas from the participant (on Heartbeat, breathing, sweating).
Designers had to prefill the sensitizing booklet to familiarize themselves with the concept of 'Social VR' and get ready.
2.2 RESULTS AND FINDINGS
[A] Degree of Interaction and Types of Entertainments
The type of entertainment would be categorized into three levels depending on the degree of interaction.
[B] Ways to Express Biosignals in Social VR
Grouping the brainstormed ideas resulted in six ways of expressing avatar biosignals in social VR.
Examples from the co-design session
2.3 CONCEPT DIRECTION
[A] Environment Setting
‘Live jazz bar’ was selected as the final environment considering the prototyping and user test feasibilities.
[B] Biosignal Visualization Types
Four out of six visualization types were selected, adjusted, and concretized to fit the context for the final evaluation.
3. Prototyping
This section introduces the technical setup and the final design prototypes of both the VR environment and the biosignal visualization.
3.1 ENVIRONMENT DESIGN
The goal was to build an aesthetically appealing environment for users to feel the real jazz band vibe when testing the prototype.
Atmosphere (Bar setting, Lighting), audience, and the jazz band (People animation, music, stage design) were considered.
[A] Atmosphere
[B] Audience and the Jazz Band
3.2 BIOSIGNAL VISUALIZATION DESIGN
[A] Biosignal Implementation
Heart rate and breathing rate were chosen as the final biosignals to be prototyped. The signals were artificially generated with a python library (NeuroKit2). Signal data was stored and mapped to each design element so that the design can animate accordingly.
Generated Biosignal Info:
-Breath (bpm): High - 21 / Rest - 14 / Low - 7
-Heart (bpm): High - 150 / Rest - 100 / Low - 50
[B] Design Implementation
Embodied
DESIGN 1
Near body particles
DESIGN 2
Around body creatures
DESIGN 3
Environment change
DESIGN 4
symbolic and straightforward
present biosignals through particles near the avatar body
a third-party identity that carries the biosignals of the main avatar
use the surroundings to express the biosignals of the avatar
WATCH SHOWCASE VIDEO 👀
4. Evaluation
This section presents the detailed process of the user study session, evaluating the effects of the final prototypes built previously. The results will be analyzed into quantitative and qualitative data.
4.1 USER STUDY SESSION
The goal was to test and compare the effect of different biosignal visualization methods in the VR jazz bar environment.
The user study session consisted of questionnaires (PART 1) and a focus group (PART 2); 16 pairs (a total of 32 people) participated in the study.
4.2 QUANTITATIVE RESULTS
The questions were answered with a 9-point Likert scale; the results were presented with box plots (IQR).
4.3 QUALITATIVE RESULTS
[A] Current Design Method
[B] Biosignal Visualization in Social VR
There were three intrinsic motivations for sharing biosignal visualizations in Social VR.
5. Conclusion
This section reflects back to the research questions raised during the research process and concludes up with listing the limitations and recommendations to the research.
5.1 REFLECTING THE RESEARCH QUESTIONS
This research started with the idea of sharing biosignals through avatars in VR to better reflect the user's current state and enhance the immersion in the VR environments.
The scope was narrowed down to the watching entertainment setting, specifically the live jazz bar scenario.
Four different biosignal visualization methods were developed, prototyped, and applied to the VR jazz bar; the user study was conducted with 32 participants to test and compare the effects.
Visualizing biosignals (heart rate, breathing rate) in watching entertainment scenarios showed possibilities of enhancing their VR experiences in three ways. Furthermore, the four design methods had their unique advantages and disadvantages; so they can be adopted, varied, and adjusted to achieve their own goals in later applications.
5.2 RECOMMENDATIONS FOR BIOSIGNAL VISUALIZATIONS
The research ends with sharing the tips and recommendations for future research and applications of visualizing avatar biosignals in the social VR spaces.
