A group of researchers developed jointly by Microsoft and NASA’s Jet Propulsion Laboratory (JPL) have been working together on Onsight, a mixed reality tool that adds an interplanetary dimension to everyday life. One day, this could be how all of us experiences space.
“My average day is that I go to the office, have my coffee, nip off to Mars for a little while, check out the latest location, write some code, and then I’m back home in time for dinner,” Alex Menzies, the software lead for augmented and virtual reality development at JPL said at the Smithsonian’s The Future is Here Festival.
The incredible OnSight software processes images taken by the Curiosity rover and makes them viewable through a HoloLens, Microsoft’s mixed reality headset. OnSight allows specialists to virtually experience the rover’s surroundings—without the price tag or physical risks of enduring Mar’s harsh conditions.
“Whenever new images get downlinked from the rover, we go through and build a full 3D scene automatically in the cloud, and that streams to the scientists’ devices,” Menzies explained. “So, they are able to put on the HoloLens and walk around on the surface of Mars shortly after the downlink is completed.”
This process is more than just fascinating. It also gives researchers the ability to plan missions and more confidently make scientific decisions. The technology allows scientists to study Maritan surface features up close, get a better idea of obstacles, and observe geological formations that would not necessarily pop out in a traditional image. The choice to with an augmented reality device rather than a virtual reality headset like Oculus allows teams to stay in the office and keep track of the data they are monitoring and analyzing.
“You can actually walk up to four kilometers in OnSight right now in a single location,” Menzies notes. “We actually build locations for every place the rover has visited, like a series of different scenes.”
“What’s really powerful about this is when you stop to think, ‘this is all real,'” he told me.
“These are all real images made from real photons bouncing off the Martian surface. This is what it is actually like to look around on another planet. Mars stops being this abstract blurry orb in the sky and it becomes a place that you can visit. You can hold meetings there. You can go on a date there if you really wanted to. That’s what is interesting to me, Mars may not have trees, and deer, and oceans, but it’s got an immense amount of different geological structures that are exciting to explore and in many ways much more extreme. The canyons are deeper, the mountains are taller, the hills are steeper, and so you get really interesting stuff.”
In a recent episode of 60 Minutes, correspondent Bill Whitaker got the opportunity to experience Mars firsthand. Walking in the Curiosity rover’s tracks, Whitaker explored the rocky Martian surface with Katie Stack Morgan, a research scientists with JPL. The area they explored is where NASA first discovered a habitable environment on Mars.
Last Fall, NASA brought OnSight to the for an exhibit at the Kennedy Space Center called “Destination: Mars.” Visitors were given their own headsets and be guided through scenes by a holographic Buzz Aldrin. That might seem futuristic, but we are actually just scratching the surface.
“We are entering a new era of space exploration, one in which we all get to go,” Menzies concludes.
Source: Motherboard, CBS