Minor Immersive Media (VR application)
Team/Project Lead, Lead Designer
During the Minor "Immersive Media" students will be assigned to one project with a random group. These projects are submitted by companies, who are part of the creative industry. The goal of these projects is to explore new technologies and create an endproduct.
In my case I was assigned to a project, which was submitted by Xsens and Metropool. They asked us to create a prototype for an algorithm, which creates precise finger and hand motions by only using an audio input of an instrument. This algorithm should then be combined with the motion capture data of a Xsens-suit to create a fully animated musician and a believable performance.
Our client from Metropool also requested a virtual representation of a Metropool stage, which can be explored with a VR headset and has the possibility to become a full product.
Role and Contribution
During the project I was chosen as the team leader and lead designer, which meant to prepare meetings with the clients and other professionals to gather the required knowledge. Additionally I tracked the overall progress of the team to finish tasks on time and to help team members in case problems came up.
In addition to my organizational tasks, I as the only designer within the group, had the task of creating concepts for the looks and interactions within the application. This meant to closely work together with the artists regarding the environment by creating concepts and to work together with the developers on the functionality inside of the application. I prepared builds for the tests, decided how to react to the test results and what should be prioritized during development.
Designing the Virtual Environment
The virtual environment had to fulfill two requirements. It was requested to be a virtual representation of a Metropool stage and should be explorable with a VR headset. To create a design that fulfills these requirements and can be expanded on to become a full product I first conducted target group research and analyzed other virtual performance applications. I then used the results to create product designs and presented them to the client.
Design 1: Venue recreation
Together with the client, we decided on the "Capsule"-design. So, my next step was to recreate the layout of the existing venue to then modify it with the "Capsule"-functionality. To do that I used Adobe XD and images, which were taken from the existing venue.
In addition to the environmental layout, I also created multiple designs for the capsules:
As mentioned in the product designs, the plan was to remove the roof and back wall of the venue to fill it with an environment fitting to the performance. So, I designed multiple possible performances and discussed them with the clients and artists.
“Dark urban” – A performance set in an urban environment at night during rain.
“Metro-Man” – A player character in the style of “Mario” moving according to the song
“Beach” – A performance set in a beach setting with an upbeat mood
“Metro” – A performance inside of an underground station
"Abstract” – A colourful background with multiple shaders moving to the song
“Pirate” – A performance set on a sailing pirate ship
User Interface design
The interface for the player changed multiple times during the project, the reason being changes or discoveries made while working on the algorithm. The first design of the interface featured the option to play as a musician or audience member. Settings to enter the capsule or leave the game were at this stage all part of a canvas, which was instantiated in front of the player on button press. To present and test the interface, I created an Adobe XD prototype, which can be tested here.
First audience interface (in game)
Later on, the musician-part of the interface was cut, since our focus shifted to the audience experience. Nonetheless, the main idea of a canvas being instantiated stayed and now also featured the option to start and stop a performance. I also prepared an Adobe XD prototype for this, which can be tested here.
Advanced audience interface
While testing we discovered that users did not like that all options are hidden inside of this canvas and they would prefer canvases inside of the venue. This would reduce the amount of steps players need to make to start a performance. So, I moved most of the options into the venue. These changes were preferred by the testers.
Simplified audience interface
New performance interface
New capsule interface
I also worked on the interaction within the application. So, I had to find a way to move the "Capsules" without causing motion sickness and additional interactions for waiting times.
A lever, which will move the capsule upwards and downwards depending on its rotation
Two buttons, which when pressed will slowly move the capsule upwards or downwards
Two buttons, which allow the player to choose between different floors
In case users had to wait for other players and to make the environment more interesting, I created multiple items in the environment.
Shaders (Unity Shader Graph)
During this project, I created multiple shaders for the performance and environment in general with the Unity shader graph. These shaders can be easily be reused in other projects as well:
In addition, I created a shader that blends colors or transparency with a noise input. This was used to dissolve the stage and to blend the colors of the light.
IClone 7 - Character Creator 3
I created the musician with Character Creator 3: