This Tuesday, August 2, a research team from the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT has revealed to the world a newly developed virtual technique that allows us to interact with objects captured in a video.
The technology is the result of an extensive research effort supported by the MIT Energy Initiative, the National Science Foundation, and the Qatar Computing Research Institute. Ph.D. students Abe Davis and Justin G. Chen headed the development of the experimental technique.
“Interactive Dynamic Video”, or IDV for short, uses video from regular cameras to record objects and detect vibrations at different frequencies. The footage then allows for virtual object manipulation based on the analyzed vibrations, as the computer algorithm simulates the movement of said object by interacting with it with a mouse pointer.
Davis will outline in his final dissertation paper all the potential uses and benefits of the technology, which already points at applications everywhere from engineering to film and virtual reality.
A Potential Level-Up for “Pokémon Go” and AR
Being aware of how something as complicated as imaging technologies and algorithms can drive attention away from revolutionary discoveries, the CSAIL research team at MIT used the massively popular mobile game “Pokémon Go” to illustrate how their technique could be a literal game-changer.
“Pokémon Go” is an augmented reality-based game that places the beloved creatures in our real world surroundings for players to capture, train, and battle. Players can detect nearby Pokémon on the app and then capture them as they appear before them in their device’s cameras.
However, players may have noted that one of the most distinct features that make the game feel “unreal” is the fact that Pokémon just appears and float around the screen. With IDV technology, the game changes as creatures would correctly spawn and interact with the environment, causing a bush to rustle, for example, similar to what the video uploaded by the research team showed.
IDV comes with unsuspected benefits
“Interactive Dynamic Video” could not only be a current AR and VR game enhancer, but it could also bring cost-effective solutions to several other industries.
Instead of developing 3-D models of a massive structure like a building or a bridge, engineers could use IDV quickly to capture video of existing constructions or prototypes for a preliminary analysis of stability and response to outer elements.
Moreover, those in the entertainment industry could also render CGI elements into video footage in a much more realistic way, as the items would appear more lifelike by interacting with their real surroundings.
This technique remains experimental, but since you would only need a camera and a computer, it shows tremendous promise for the future of virtual reality and other professional fields. At its earliest phase, IDV has proved to be an accessible, versatile, and powerful technology.
Source: MIT News
