Disney develops new technology at a near constant pace, and what projects they are working on give us not only an idea of their invention process, but also a glimpse into what we may eventually see from them in the future. Patents filed by Disney Enterprises, Inc. this week cover a range of animation applications as well as some technology to improve immersive storytelling in virtual reality environments.
Here is what’s new on the Disney animation front:
“Real-Time High-Quality Facial Performance Capture”: This patent focuses on recording an actor’s facial expressions in real-time, while also capturing the “finer-scale details” of the face, such as wrinkles or moles. While most current facial capture tech involves multiple cameras, elements of this system may only require one. The patent states, “Although wrinkles appear at different locations on the face and with different orientations and scales, their visual appearance can be a direct consequence of the shading caused by their local shape. Embodiments of the disclosure can learn the relationship between local image appearance and local wrinkle formation and use such information to reconstruct plausible face wrinkles in real-time from a single RGB camera.”
The document goes in-depth on the details and applications of the process, which are too involved to describe in full. An important thing to note though, is that Disney — along with Fox and Paramount — has been involved in an ongoing lawsuit regarding their use of the MOVA motion capture facial recognition software developed by Rearden LLC while creating a number of their films. Not relying on outside sources for motion capture technology would help them avoid these situations in the future.
“Systems and Method for Virtual Entity Animation”: This proposes a solution for certain problems that arise when using a virtual skeleton covered in a virtual “skin” for character animation purposes.
“Point Cloud Noise and Outlier Removal for Image-Based 3D Reconstruction”: Current technology has the ability to take two-dimensional images and transform them into three-dimensional formats. It isn’t perfect though, and sometimes problems and inconsistencies have to be smoothed out, and that eliminates the finer details of the scenery. This patent aims to fix that.
“Deep-Learning Motion Priors for Full-Body Performance Capture in Real-Time”: Capturing an actor’s body movements for later animation can be tricky business. Generally, it involves the actor performing a physical task with numerous markers attached to their body. This patent proposes motion priors to fill in the blanks caused by dropped or malfunctioning markers, as well as using separate spatial and temporal filters.
Virtual reality was another heavily covered topic this week; here are the recent developments in that area:
“Virtual Blaster”: This ambiguously-named patent actually covers a large range of immersive gaming experiences in and out of virtual reality, by combining a game system with other electronic devices. The system would not only sense muscle movement to properly render a player’s character in a virtual environment (such as a player performing a movement to cast a “frost spell” at the controller device, then seeing the spell fly and his opponent freeze in place), but could also interact with their at-home environment to add to the player’s enjoyment — adjusting lighting for better mood, or sound for better immersion.
Here is just one example from the patent: “For example, in a story that simulates the user being within a bee hive, the various storytelling devices could each output the sound of a bee buzzing with a time delay in between the output of each of the devices, such that the sound of the bee buzzing appears to move throughout the physical environment. Thus, if the user is positioned between the various storytelling devices used in the playback of the story, the sound of the bee buzzing could appear to encircle the user, thus creating an immersive auditory experience for the user and enhancing the realism of the story.”
The document also mentions combining the system with a user’s mobile device, so that projections on their phone’s screen would become visible in the augmented reality.
“Elastic Tension Changing Headband”: This aptly-named patent is for a virtual reality headset headband; one which changes tension, and is also elastic.
“Displaying Proximate Keys in a Virtual Reality Environment”: Lastly, this patent aims to grant visually-impaired users, such as ones wearing a VR headset, the ability to interact with other devices (like pressing a button or typing in a code in real life) that they can’t actually see, while also granting them a visual simulation of the device they are using.
Source/Images: U.S. Patent and Trademark Office