Logo (1).png

Project:

Duration:

Clients:

Software:

Languages:

Primary Role(s):

Team-size:

Minor Immersive Media (VR application)

18 Weeks

Metropool, Xsens 

Unity

C#

Team/Project Lead, Lead Designer

6

During the Minor "Immersive Media" students will be assigned to one project with a random group. These projects are submitted by companies, who are part of the creative industry. The goal of these projects is to explore new technologies and create an endproduct.

In my case I was assigned to a project, which was submitted by Xsens and Metropool. They asked us to create a prototype for an algorithm, which creates precise finger and hand motions by only using an audio input of an instrument. This algorithm should then be combined with the motion capture data of a Xsens-suit to create a fully animated musician and a believable performance.

Our client from Metropool also requested a virtual representation of a Metropool stage, which can be explored with a VR headset and has the possibility to become a full product.

Role and Contribution

During the project I was chosen as the team leader and lead designer, which meant to prepare meetings with the clients and other professionals to gather the required knowledge. Additionally I tracked the overall progress of the team to finish tasks on time and to help team members in case problems came up.

In addition to my organizational tasks, I as the only designer within the group, had the task of creating concepts for the looks and interactions within the application. This meant to closely work together with the artists regarding the environment by creating concepts and to work together with the developers on the functionality inside of the application. I prepared builds for the tests, decided how to react to the test results and what should be prioritized during development.

Designing the Virtual Environment

The virtual environment had to fulfill two requirements. It was requested to be a virtual representation of a Metropool stage and should be explorable with a VR headset. To create a design that fulfills these requirements and can be expanded on to become a full product I first conducted target group research and analyzed other virtual performance applications. I then used the results to create product designs and presented them to the client.

Product Designs


Design 1: Venue recreation This design focuses on recreating the stage. In this design, the musician will have the stage as a playground, while the audience can move around the venue. Users, who visited a Metropool venue before, would then be able to recognize the layout. This on the other hand would only apply to users, who have visited the area before. Additionally, it limits the customizability of the environment and the "virtual seating"-area, which can negatively impact the user experience. Design 2: "Cinema" Built on the first idea this design would still use the stage layout but will remove the roof and wall behind the stage. This empty space can then be filled with a customizable environment, while still maintaining the Metropool layout. This customizable environment will then change depending on the performance. Design 3: "Dome" To also present the option of a new layout I created a design, which features a dome-shaped design with a stage in the center. The outside of this venue will, like the previous design, change depending on the performance and will be fully customizable. This design will give users a lot of freedom in choosing a seat angle. Nonetheless, the connection to the existing venue will be less clear. Design 4: "Flying audience" With the layout of the second design, this design additionally features a new way to move through the venue. The idea was to make the users smaller and to give them a vehicle to fly. This would give users way more freedom in selecting a seat position and the new movement gives opportunities for gamification elements. The issue here is that these vehicles will require skill to use and have a high chance of causing motion sickness. Design 5: "Capsules" To solve the issues of the previous approach, this design was created. The idea is to limit the movement of the vehicles to reduce the chance of motion sickness. These vehicles or "capsules" will then function as small lobbies, which can be entered. "Capsules" will be placed around a spherical surface around the stage and can be moved vertically to cause the least amount of motion sickness possible.




Scene progress


Together with the client, we decided on the "Capsule"-design. So, my next step was to recreate the layout of the existing venue to then modify it with the "Capsule"-functionality. To do that I used Adobe XD and images, which were taken from the existing venue. Layout of the existing venue Modified layout After the layout-design was approved I proceeded to list all required environmental assets with reference images to an asset list. Additionally, I prepared a graybox-scene within Unity to give the artists a better reference and our clients a better idea of the looks. Graybox-scene Over the next weeks in the project I proceeded to implement the assets, which were prepared by the artist. Additionally, I worked on the post processing and lighting of the scene. Scene progress (week 7) Final scene (week 15)




Capsule Design


In addition to the environmental layout, I also created multiple designs for the capsules: First Design Multiple designs Final design (Modelled by Martynas Žilinskas)




Performance Design


As mentioned in the product designs, the plan was to remove the roof and back wall of the venue to fill it with an environment fitting to the performance. So, I designed multiple possible performances and discussed them with the clients and artists. Overview of performance designs:

  • “Dark urban” – A performance set in an urban environment at night during rain.
  • “Metro-Man” – A player character in the style of “Mario” moving according to the song
  • “Beach” – A performance set in a beach setting with an upbeat mood
  • “Metro” – A performance inside of an underground station
  • "Abstract” – A colourful background with multiple shaders moving to the song
  • “Pirate” – A performance set on a sailing pirate ship
Together we settled on the "Dark urban"-performance, which was then produced by the artist and me:





User Interface design

The interface for the player changed multiple times during the project, the reason being changes or discoveries made while working on the algorithm. The first design of the interface featured the option to play as a musician or audience member. Settings to enter the capsule or leave the game were at this stage all part of a canvas, which was instantiated in front of the player on button press. To present and test the interface, I created an Adobe XD prototype, which can be tested here.

early.png

First audience interface (in game)

Later on, the musician-part of the interface was cut, since our focus shifted to the audience experience. Nonetheless, the main idea of a canvas being instantiated stayed and now also featured the option to start and stop a performance. I also prepared an Adobe XD prototype for this, which can be tested here.

ea1d71279379351982be6b202091b850.png

Advanced audience interface

While testing we discovered that users did not like that all options are hidden inside of this canvas and they would prefer canvases inside of the venue. This would reduce the amount of steps players need to make to start a performance. So, I moved most of the options into the venue. These changes were preferred by the testers.

06155e3959305f24dfadcf954ad46af5.png

Simplified audience interface

PerformanceUI (2).png

New performance interface

9fbc2a9bdef3c53ad3a0ed32c9304e4e.png

New capsule interface

VR Interactions

I also worked on the interaction within the application. So, I had to find a way to move the "Capsules" without causing motion sickness and additional interactions for waiting times.

Capsule movement


As previously presented, the goal was to allow the player to move the "Capsules" vertically. To find a way to move the capsules with the smallest risk for motion sickness, I created prototypes for three different ways to move the capsule. The following ways were testable:

  • A lever, which will move the capsule upwards and downwards depending on its rotation
  • Two buttons, which when pressed will slowly move the capsule upwards or downwards
  • Two buttons, which allow the player to choose between different floors
Through A/B-testing we managed to find out that the buttons, which need to stay pressed were favored, so their design was implemented in the final prototype.




Additional functionality


In case users had to wait for other players and to make the environment more interesting, I created multiple items in the environment. Overview: Camera A camera, which creates a .png of its target render texture: Bottle + Drink dispenser A bottle, which enables and disables a particle effect depending on its rotation. A drink dispenser, which can be used to recolour glowsticks and bottles: Glowstick A glowstick, which can be turned on and off: Doors Doors that can be opened and closed: Spray can A spray can that uses a ray cast to determine a hit location, where a prefab of a plane with a Metropool-logo, is instantiated:





Additional Contributions

Shaders (Unity Shader Graph)


During this project, I created multiple shaders for the performance and environment in general with the Unity shader graph. These shaders can be easily be reused in other projects as well: Rain shader This is a shader I created for the stage during the performance. I combined it with rain effects, which I created in the Unity particle system. This shader also uses tri-planar mapping to make the rain impact appear on the top of objects and raindrops on the side of objects: Fade shader Fade shader
In addition, I created a shader that blends colors or transparency with a noise input. This was used to dissolve the stage and to blend the colors of the light. Stage dissolve Light blend Bottle shader To create a more beliveable bottle, I also created a shader that simulates the liquid inside of the bottles. Building shader To make the buildings feel less static I created a simple shader, which creates a pattern with a voronoi node.




IClone 7 - Character Creator 3


I created the musician with Character Creator 3: