View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

Google Daydream tech makes VR headsets ‘invisible’

New Google technology hopes to 'remove' the headsets from VR and MR experiences.

By Joe Clark

Google’s new Mixed Reality tech aims to create a more communal experience by removing the obfuscating elements of VR headsets.

Google Research and Google Daydream Labs say that the new the new tech will overlay VR headsets with the users face, giving the experience a much more human touch. Currently VR is great for the person inside, but spectators can have a hard time trying to gauge what’s actually happening.

The new tech reportedly uses a combination of machine learning, 3D computer vision, and advanced rendering techniques to ostensibly “remove” the headset obscuring the users face.

google daydream vr

Via Google Blog: VR user captured in front of a green-screen is blended with the virtual environment to generate the MR output: Traditional MR output has the user face occluded, while Google’s result reveals the face. Note how the headset is modified with a marker to aid tracking.

Avneesh Sud, Software Engineer, Google Research, said: “If you’re watching someone else use VR, it’s hard to tell what’s going on and what they’re seeing. And if you’re in VR with someone else, there aren’t easy ways to see their facial expressions without an avatar representation.”

“Mixed reality is a way to convey what’s happening inside and outside a virtual place in a two-dimensional format. With this new technology, we’re able to make a more complete picture of the person in VR.”

The way the technology works is by creating a 3D dynamic model of the users face to accurately portray any facial expressions or emotions, and then layer it onto the front of the digital headset. This allows spectators to see the users face onscreen despite it being hidden under a VR helmet.

The virtual world is then rendered on a green screen, whilst the video feed of the rest of the body is captured with a camera. For anyone watching on another screen the result is a complete version of the user and the world they’re inhabiting, with their face free of obstruction.

Content from our partners
Green for go: Transforming trade in the UK
Manufacturers are switching to personalised customer experience amid fierce competition
How many ends in end-to-end service orchestration?

The technology is a long way from public release however, and the requirement of a video camera, a green screen, and a VR headset itself is a high barrier to entry. The tech is also not currently running on Google Daydream and instead uses a modified HTC Vive.

Sud said: “Our initial work focused on mixed reality is just one potential application of this technology,” adding “It’s just the beginning for this technology.”


Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.