Inside PETA "Eye To Eye": Virtual Reality Live-Acting Experience
After winning an FWA Of The Day for PETA "Eye to Eye", the VR experience we created together with Kolle Rebbe, the team asked us to share some insights with their audience. And here we go...
What did you want to accomplish with the project?
In collaboration with Kolle Rebbe and PETA we sought to create an uniquely emotional virtual reality experience that allows users to engage in real-time dialogue with a virtual animal, and observe the world from its perspective.
Our idea was to develop an unparalleled and immersive application. The purpose was not only to bring out the sensitivity in people, but also to put them in their counterpart’s shoes - and evoke a deeper emotional feeling. As the world’s largest animal rights organisation, PETA should provide a new, eye-catching and compelling way to reach out to people who have little or no previous involvement in animal rights - and create an experience that is most definitely at eye-level with the users.
What technical challenges did you encounter and how did you solve them?
The biggest challenge was the hardware setup in combination with a flawless character design in the application, combining visual excellence and real-time interaction: A motion-capture setup transmits the voice, facial expressions and gestures of an actor into the modelled character - a rabbit - who in return reflects the gestures of the actor. The actor’s voice is transmitted into the virtual world and triggers the dialogue with the respective user. Applying the anatomy and gestures of a human on a convincing 3D object in real-time was quite a challenge - a mixture of tried and tested tracking software and a custom hardware set-up with continuous refinements were our way to go. Last, but not least: The hairworks and the level of detail of our virtual rabbit are something, we’re really proud of as well. This component took some 500 hours of 3D Art and Creative Engineering.
What did winning the FWA award mean to you?
The final release of the application was quite intense. So it was really great for us, to receive immediate feedback within the launch phase. While we were still working on the mobile VR version, the event setup got amazing feedback: On average, people spent 12 minutes in the experience. At the same time, winning an FWA after just one day of voting was the cherry on our vegan cake. It pushed us forward within the final project phase and helped to make people all over the world aware of this experience. What more can you ask for...
Tools used:
A crucial part of the setup is the stable and performant motion capturing. For body-tracking, we used Perception Neuron by Noitom. The face-tracking is done by FaceWare Live. For the complex modeling of character and environment, we mainly used Maya, ZBrush, 3ds Max as well as TopoGun. Texturing has been done with Mari, Substance Painter, Photoshop and Knald. The grooming process of the main character hairworks has been implemented with Peregrine labs Yeti. As a sound engine, our team used FMOD, while sound editing has been done with Nuendo. Last, but not least, the application was built in the 3D Engine Unreal with NVIDIA HairWorks and VRWorks as additional components.
Three hot facts:
- Our experiment made hundreds of people reconsider their treatment towards animals already. Besides it had an impact on our colleagues: Some of them decided to live vegetarian, among them our sound engineer who reviewed some 200 hours of video and audio of animal abuse content to deliver realistic SFX.
- After release, many VIP supporters of PETA tried out the application, among them Germany's biggest "Let's Play" Influencer UNGE. Besides it became trending topic on Youtube, the videos dealing with the case have more than 2 million views already, and it received great feedback from media worldwide. Fun fact: Among the test persons was a German actor named Sky DuMont. As we always use to say: Sky’s the limit.
- The experience was launched at re:publica '18 in Berlin and is about to be presented at a lot of further occasions. Festivals, trade shows, we do hope, that in the long run, we helped to make people aware of an important issue in a really new and deeply connecting manner.
