Published on Tuesday, 10 September 2013

 

ToyTalk Talks Back with Interactive Talk Show App

ToyTalk is a new company that creates entertainment through characters and conversation. Their first project is called ‘The Winston Show’, an iPad app that works like a highly interactive talk show that features the player, who can participate in the show while the characters listen and talk back. On the set, players can chat with Winston about favourite topics, test their knowledge on a quiz show, play backstage in the costume department and choose from an ever-changing list of stories to listen to and talk about.
Toytalk-winston-show

The Interactive Difference

The company's CEO Oren Jacob and CTO Martin Reddy started Toy Talk after both having worked for many years at Pixar on animated feature films. Bobby Podesta took the role of director and oversaw the animation for The Winston Show project. He organised the creation of the iPad app similarly to the way a producer might set up the production of an animated movie, using a recording booth, voice talent, animators, modellers, writers and so on. The character animation is a mix of CGI and 2D animation.

However, he has had to carry out several tasks differently. For example, once a movie has been delivered, the project is complete and the work and development come to an end. But in this case, the app is being introduced to the market with an initial number of responses and subjects that the characters and player can discuss, however, that will continue to grow and change each week depending on what audiences find interesting. 

 Toytalk-winston-show Toytalk-winston-show

That means that for the first time, Bobby and the team have had to figure out a way to create animated characters that can lip sync thousands of lines of dialogue, all of which are subject to change. Furthermore, the characters will have to respond depending on what players are talking to them about.

In order to do this, they created a new lip synching technique and system. The app’s software is original, written in-house over an 18 month period by a team of engineers at ToyTalk. It is based on speech recognition linked to a large database of animation options for Winston, to provide the app’s interactivity. It also allows Winston to acquire a progressively larger range of responses. “With more than six hours of content in the App, we knew it would be too much dialog to animate all the lip sync by hand.  With this in mind, we developed a way to take the show’s character dialog, extract the phonemes from the audio performance, and assign mouth shapes that would then be animated programatically. 

 

Pulling Strings

“Behind the app, an internal tool called PullString authors the content for our AI engine. This app can detect when new content is available on our servers and automatically download it so that the character can effectively grow and learn over time. PullString also triggers animation, audio recordings and sound effects for the characters, and is able to remember everything that has been said, so that it can avoid repeating itself. The responses that the characters make to kids can be personalized based on various criteria like time of day, how long they've played, and even conversations that they've had earlier.”

As mentioned, members of the ToyTalk development team have the same roles as an animation production team on a movie or a TV show, but this app continues, becoming an opportunity to work on layers of entertainment instead of a linear story. During development, the story has remained the focus of The Winston Show, and is more important than looks or sophistication of the animation.

 Toytalk-winston-show5

Bobby said, “We used storyboards to define and pitch the look, feel and interaction of The Winston Show.  The storyboards revealed the characters for the first time, with acting beats and dialog, and were used as a blueprint for what we were to build, creatively and technically.” However, with the ability to constantly iterate and add animation via the cloud, the team also has the ability to continue to improve and refine the look and performance of the characters over time.

The show currently has two starring characters, the host Winston and his sidekick Ellington, as well as a dozen or so other featured characters. There is one microphone button and the show has been designed for one player, but they have found in early testing that multiple viewers enjoy playing as well. ‘The Winston Show’ is available now on the App Store.  www.toytalk.com