New Disney Research Project Translates Written Scripts Into Virtual Reality Previews

Whether live action or animated, the finished story you see on screen takes countless hours of effort before it’s fully formed. Developing a written script into an engaging visual concept can be daunting; Disney Research is working on a way to streamline the process.

During San Francisco’s Virtual Reality Developers Conference on Monday, Disney Research digital platforms group lead Sasha Schriber described Project Cardinal, which would convert natural language scripts into working storyboards and visualizations. “It goes from script to storyboard to animation in real-time,” Schriber explained.

Using a written script, Cardinal creates a virtual reality pre-visualization. Filmmakers use roughly animated pre-visualizations to get a better perspective on how characters and settings will actually look and behave once filming or animating is underway, and how any changes in the script will affect the scene visually. This, combined with the initial storyboarding of a script, can significantly delay a project. Of storyboarding and pre-visualizations, Schriber said, “This is a very lengthy process. Some animation films take sometimes up to two years to finalize the story.”

By converting scripts into virtual reality animations, filmmakers can preview, change, and manipulate aspects of the story as if they were really there, a simulation of their natural vantage point — working with actors on a set. Cardinal also allows for immediate onsite integration of voice recordings to the animation. Disney Research is working towards adding advanced editing capabilities to the virtual reality animation, as well.

Project Cardinal is undergoing testing with VR filmmakers at the moment, and is facing a key issue. Script writers use different styles, so the natural language processing involved can become difficult. To combat this, Disney Research has advised script writers to streamline their work, stick to some constants (like only using present tense), and utilize a simplified markup language they have developed.

Although it is currently relegated to VR filmmakers, Schriber explained that Disney Research aims to make this technology viable for traditional filmmakers as well.

Source: Variety