Virtual Reality Live Acting Experience
Using interactive technology such as motion capture and live acting, we developed a uniquely emotional virtual reality experience for PETA that allows users to engage in real-time dialogue with a virtual animal, and observe the world from its perspective.
A brand new use-case in VR
We have conceived the idea and use-case together with Kolle Rebbe, and have managed to develop an unparalleled and immersive virtual reality application. The purpose of this experience is not only to bring out the sensitivity in people, but also to put them in their counterpart’s shoes - and evoke a deeper emotional feeling. As the world’s largest animal rights organisation, PETA has found a new, eye-catching and compelling way to reach out to people who have little or no previous involvement in animal rights - and this experience is most definitely one that is at eye-level with the users.
Enhancing learning through technology
The main idea is simple: a simple conversation can work wonders. However, a dialogue with an animal offers an unprecedented communication platform for PETA. The detailed and real-time virtual reality applications used made it possible to enhance the quality, and therefore allow users to be fully immersive within the animal’s perspective.
Live Acting in Virtual Reality
Simplicity is created with the use of a complex technological setup: A motion-capture setup transmits the voice, facial expressions and gestures of a real-time actor into the modelled character - in this case a rabbit - who in return reflects the gestures of the actor. The actor’s voice is transmitted into the virtual world and triggers the dialogue with the respective user. The special feature: users have no idea that their counterpart is controlled by an actor. This convinces them that the conversation they are having with the virtual animal is true. Through this approach, the application offers an exceptional level of detail while offering a non-linear and unique experience.
Development of a unique touchpoint
The combination of both conceptual and technological approaches offers a completely new touchpoint for PETA in order to convincingly draw attention to animal rights, while at the same time meeting the organisation’s standards and demands.
Real-time framework in Unreal
In order to achieve this, we have used a real-time framework that has been developed in Unreal - a game engine - and have explored a variety of possible solutions for face and body tracking, real-time 3D animation as well as binaural sound design.
Unique setup
This unique and high-performance setup which was developed in six months is stable enough to be used at festivals and events - reaching out to a larger audience.
Complex motion and face tracking
The experience was made possible with the use of a multitude of technical components which posed new challenges to the concept and creative engineers, as well as our 3D artists - who required a dedicated amount of time during their exploration phase. In parallel to VR, the core of this experience is face and body tracking. Facial recognition software captured the actor’s facial expressions and transmitted them in real-time to the 3D model: the virtual animal. As a dialogue partner, the rabbit represents the suffering of all animals, and reaches out to the logo of the animal rights organisation.
Environment design in Unreal
For the background and surrounding, five different worlds were conceptualised, modelled and developed in the Unreal Engine. Abstract themes - such as the value of life, freedom, animal experiments and the handling of animals in the food industry - have allowed us to portray a scenario-based tangible environment. The user is then guided through the scenarios by the virtual persona. Immersion becomes particularly emphatic due to the minimalistic set design, which is also found in the virtual world - hence creating an absolutely fluid transition between real and virtual reality. During the concept work, user interfaces were deliberately omitted in order to reduce the artificialness.
Perfection in 3D-design and animation
The look-and-feel of the scenarios and of the character have been constantly developed through the course of the project. The main focus was to generate maximum attention to detail, authenticity and deep realism in order to keep the dialogue between the user and the virtual character as convincing and long-lasting as possible. The application should appear as cinematic as possible. The rabbit was designed internally from scratch and was placed in the centre of the project. One of the main requirements was that the rabbit would be able to reproduce the movement and facial expressions of the actor in high detail. From the skeleton, the facial expressions and blending shapes, to the highest possible level of detail of skin, eyes and hair, our team of 3D artists and art directors have spent over 500 working hours and have managed to create a figure and a persona that has set very high standards.
Immersion through sound design for VR
In addition to visual components, a dynamic and binaural sound design is an extremely significant factor for an immersive virtual reality experience. This was realised by the German Wahnsinn recording studio. An auditory isolation of the outside world has been setup and has created a complete immersion in the scenarios with the use of application-specific 3D sound. Our team of experienced sound engineers have allocated great importance to the combination of real animal sounds and sound effects that have been adapted to the technology and the use case. Throughout the development and during user testing, the sounds have been constantly expanded and refined. Lastly, since the application will be experienced during events and festivals, the combination of sound isolation of the set-up as well as sophisticated SFX is essential.
Mobile VR – from a tour truck to your smartphone
The installation has a unique setting: inside a mobile truck, actors and users interact without even meeting each other. The perfect interaction of all elements, from tracking the actor, to live rendering of the character and enhanced sound effects - is what will define the success of the application. At several events such as re:publica and other festivals, the live installation will attract a lot of attention and will be a true magnet for visitors. A WebVR application for smartphones which has been adapted to this basis will subsequently achieve a wider reach and will make the topic accessible worldwide.
Results
The application sets new standards, and not only for virtual technology or NGOs. It offers a unique tool that can be used as a communication platform like never seen before. The experience is different from one user to another - it is individually tailored - which explains the reason why the User Testings lasted on average over 10 minutes per user: a new benchmark for VR.
