Assasins Creed - FACING THE ASSASSIN

Published on Wednesday, 24 March 2010

‘Assassin’s Creed II’ is now on the loose, released in November. The game’s publisher assisins-creed
Ubisoft has poured resources into the new IP, both to create it and to promote it, giving
some well-known effects companies a chance to apply their skills to several related projects.

 

assasin creed
assasin creed 10 assasin creed 11 assasin creed 5
Ubisoft acquired Canadian VFX house Hybride to produce three short films generating further interest in the ‘Assassin’s Creed’ story. They were shot on entirely green screen sets, then environments from the game were placed into the backgrounds. The characters, script, fight choreography and assets from the game and the films were all closely aligned. Digital Domain produced a high profile TVC for the IP as well, again borrowing looks, characters and scenes from the game and applying a macabre twist.

Believable Faces
Image Metrics was also heavily involved, providing facial animation for a number of scenes and 23 different characters. Although the identity of lead character Ezio Auditore da Firenze is kept shrouded in a hood, Ubisoft wanted to develop believable facial animation for key supporting characters in the game, including fellow assassins, priests, ladies and artists.
Starting with performance capture and character rigs from Ubisoft Montreal, Image Metrics optimised the rigs and used character specific reference data to deliver 28 minutes of facial animation. A team comprising four image analysis specialists and ten animators were involved for the production cycle of the job, plus two production managers. Using their proprietary, performance-based facial animation and rigging system, they completed the job in only about eight weeks.
Toward the end of 2008, several iterations passed back and forth between the two studios to improve the assets. This involved optimising the rigs’ design to work best within the Image Metrics workflow. “It was easy for us to work within the Ubisoft pipeline,” said Peter Busch, Image Metrics Technical Account Director. “Because our facial animation system analyses video of real actors performing the CG roles for accurate motion down to the pixel level, most of the iterations were done to allow our animation team more localised areas of control to better match the performances of the ‘Assassins Creed II’ actors.”

Performance-based System
“In ‘Assassin’s Creed II’, some of the talent cast for facial animation was different than the body talent cast for each character, a fairly common scenario in game development. Our challenge was to seamlessly combine the two performances,” Peter said.
Their main concern is always the actual performance captured on video and used to drive the rigs in their facial animation system. “Because our technology largely depends on the actual video performance we feed into it, we make sure we get the best talent in front of camera. The results in ‘Assassin’s Creed II’ were very lifelike, visually pleasing and true to the time period. Ubisoft even went as so far as to have an Italian voice coach on the VO booth recording sessions to make sure the proper dialect was used in the deliveries.”
Their proprietary software could match up the visual look of the dialect as well. Because Image Metrics work with lots of different studios, they can use best practice to get accurate animation out of almost any asset the client provides, or help improve the asset based on the client’s game engine or hardware.

Intensive Research
The Ubisoft Montreal rigging team provided them with 23 unique rigs for the characters. They were all based on one master underlying control set but in most cases, special tweaking to the skinning and weighting was done to give each character individual traits. The software allowed a smooth transition when translating each performance back onto its corresponding character rig.
Ubisoft had thoroughly researched the project prior to Image Metrics’ involvement. Their team provided plenty of reference material for each character. “We got a script for the game and concept art per character with a comprehensive, three-page character bio,” Peter said. “These bios served as our bibles for the whole production period, covering personality traits, history, background and even family roots. Our team was well versed on each character’s identity before even one keyframe of animation was set. The software is designed to capture these key elements so that we can build them into our pipeline and give them the animation curves needed to deliver the performance based on Ubisoft’s artistic direction.”

Creative Expectations

Peter explained that because the two main inputs into their system - character rigs and video – determine the quality and speed of the results, Image Metrics emphasises the optimisation of these two factors. “We also make sure we are completely clear on client expectations prior to animation. In this case, Ubisoft had given us their character style guides and basic reference poses of each character, exploring each one’s core emotions - who they are, how they react, and what their underlying personality is.
“Once we had the look of the character nailed and pre-qualified the ‘Assassin’s Creed II’ character rigs, we shifted our focus to the performance capture video. We could have a brilliant rig, but if the performance isn’t there, it will show. In ‘Assassin’s Creed II’, the acting talent was tremendous and full of life, which quickly translated to brilliant performances on well-built rigs.”
assasin creed 13 assasins creed 17 Assasins Creed 16

Facial Animation Workflow at Image Metrics
Peter Busch explains the process:
In a typical VO booth session, clean, evenly lit views of this performer were captured. After the performance was recorded on video, the team put the footage through our proprietary image analysis process. This process performs two passes on every pixel on every frame of video.

The first pass uses general knowledge of the human face - distance from the bridge of your nose to the top of your eyebrows, the tip of your nose to the top of your lip and so on - to build a rough 3D model from a 2D reference image. We ‘teach’ the system which pixels are eye pixels, mouth pixels, etc. From this, a rough pass is constructed of a human face, nailing major movements in the jaw, lip sync and eyebrows.

The first pass uses general knowledge of the human face - distance from the bridge of your nose to the top of your eyebrows, the tip of your nose to the top of your lip and so on - to build a rough 3D model from a 2D reference image. We ‘teach’ the system which pixels are eye pixels, mouth pixels, etc. From this, a rough pass is constructed of a human face, nailing major movements in the jaw, lip sync and eyebrows.
o begin on ‘Assassin’s Creed II’, animators used the entire reference library to ‘pose’ the characters into extreme expressions and phonemes. Our artist-driven software relies on our facial animation experts’ trained eye to devise creative poses based on the reference. These poses ‘teach’ the system how a smile, the sound ‘M’ and other moves look. These 3D poses on the character rig, combined with the image analysis, gave our animation team their results very quickly.

Because the technology is iterative, poses may be added to improve lip sync, add more asymmetry and other refinements. The image above shows our internal retargeting technology from a sample file of typical poses set for animation. Note that the team breaks each performance out into major ‘pose groups’, creating explicit poses for the brows, eyes and mouth.

Once the results were in place, our artist’s animation is 80% complete with core timings and, hardest of all, lip sync nearly nailed. The team then pushes the performance, guided by Ubisoft, to really sweeten it, sometimes exceeding what was captured on video and heightening the entertainment value. A series of proprietary tools cleaned and ‘pruned’ the keyframe data to resemble hand-keyframe animation curves. The tools scanned keyframe patterns and reduced the amount of data, discarding redundant keys. Changes could be made continuously to the data, which remained ‘live’ and was never baked back to the joints or controls. This data was then delivered to Ubisoft’s animation team to further enhance the performance and adjust eyelines for actors recorded separately in the VO booth.

Words: Adriene Hurst
IMAGES: COURTESY OF IMAGE METRICS

Featured in Digital Media World. Subscribe to the print edition of the magazine and receive the full story with all the images delivered to you.Only$77 per year.
PDF version only $27 per year
subscribe