Microsoft HoloLens will soon be in the hands of developers, a significant milestone in the story of the augmented reality headset. But HoloLens has already broken barriers that most other devices can only dream of. Indeed, last year, the headset embarked on its first voyage to the final frontier, joining astronauts aboard the International Space Station (ISS).
Microsoft joined forces with NASA on Project Sidekick, in an effort to explore how HoloLens could provide virtual assistance to astronauts aboard the station. Last June, Sam Scimemi, NASA director of the ISS program, said that "HoloLens and other virtual and mixed reality devices are cutting edge technologies that could help drive future exploration and provide new capabilities to the men and women conducting critical science on the International Space Station."
Today, NASA published a short video overview, giving a flavor of how HoloLens was put to use aboard the ISS:
Two of the headsets were sent into space in December 2015, and were used in two different ways, as NASA explained:
Sidekick has two modes of operation. The first is “Remote Expert Mode,” which uses Skype, part of Microsoft, to allow a ground operator to see what a crew member sees, provide real-time guidance, and draw annotations into the crew member’s environment to coach him or her through a task. Until now, crew members have relied on written and voice instructions when performing complex repair tasks or experiments.
The second mode is “Procedure Mode,” which augments standalone procedures with animated holographic illustrations displayed on top of the objects with which the crew is interacting. This capability could lessen the amount of training that future crews will require and could be an invaluable resource for missions deep into our solar system, where communication delays complicate difficult operations.
On February 20, 2016, Commander Scott Kelly, senior officer on ISS Expedition 46, carried out the first Project Sidekick checkouts in space, and made the very first Skype call from the station to Mission Control.
Commander Kelly recently returned to Earth, and as you can see in the video above, he said of his experience with HoloLens: "We messed around with it for, like, two hours, and immediately I sensed this is a capability we could use right now."
Project Sidekick is just one component of a wider partnership between Microsoft and NASA to explore how holographic computing may be able to help in space exploration. ISS director Scimemi said that technology such as this "could also empower future explorers requiring greater autonomy on the journey to Mars."