Although virtual reality can transport a person to amazing locations with imaginative scenarios, it doesn"t give onlookers much of an experience. While there are setups that allow those watching the interaction to see what is being done in the virtual space, there is still the lack of emotion from the wearer that cannot be shown, due to the mask that is physically covering the face and its expressions.
Google Research has now teamed up with Daydream Labs to "remove" the headset from a VR experience so that onlookers can actually see facial expressions in real time.
In order to accomplish this, a 3D model of a user"s face was captured, including changes in facial appearance due to eye movement and blinking. Next, a specialized HTC Vive headset, altered with special eye-tracking equipment is used to capture the user"s gaze. Lastly, the 3D model is blended with the eye tracking data to create a composite image in the virtual world where onlookers can see accurate expressions.
Naturally, this is still in its early stages but it is an interesting concept nonetheless.
Source: Google