Researchers and PhD candidates at MIT have managed to create a very cool piece of technology that could make our mixed reality interactions much more believable. In other words, you might soon think that Pokémon are actually in front of you.
The researchers managed to create an algorithm that analyzes video of moving objects and determines their general characteristics. Based on that, the program can then determine the way an object might move when touched, or prodded, or nudged by the wind, and create a virtual representation of that. That means with only a few seconds of footage, a computer may know how an object realistically moves, making interacting with it digitally much more believable.
The technology is very impressive and it might have quite a number of applications from engineering to architecture, to material design, to graphics and special effects. However, one application that’s simple to understand would be in mixed, or augmented, reality scenarios.
For example, when playing Pokémon Go, the Pokémon that you see on the screen doesn’t properly interact with the world around them. Using this tech, they might start to seemingly have an effect on the real world where they show up. Another, more advanced scenario, could be on something like the HoloLens. Microsoft’s advanced holographic headset already knows how to map out the 3D world around it, but couple with this tech from MIT it might soon learn to identify the material around it and how they might realistically react when a digital character moves on a real surface.
As you can see in the video, the technology, while impressive, still has a long ways to go before it shows up in other fields, but here’s hoping that happens sooner rather than later.