Game Character Creation Series: Kila Chapter 7 – Facial Rigging with Joints

When it comes to animation, the ability to move and pose your character is essential when portraying an action or emotion. Body movement is one thing, but the character could still feel lifeless with a blank expression on its face. For your hero or heroine to truly emote and connect with the player, facial animation is crucial.

There are numerous ways to approach facial animation for games, but in this series we will focus on the two which are most widely used. These are either a joint-based, or morph target–based solution.

Either approach has its benefits over the other, but whichever you opt to use on future projects could entirely come down to the game engine you are using, or even just the preference of the developer.

In this tutorial we will look at the joint-based approach, with the morph target (or blend shape) alternative coming in Part 8.

Additional Files:

• Download the Project Files for Part 7.

Joint-Based Rig

In essence a joint-based rig is just that. A series of joints is placed around the face and then connected to the model. These can then be moved, pulling the face around to form a multitude of expressions and poses.

If you have been following along with this tutorial series you will know that Kila, the character which has been the focus of the series, is now at a stage where she has a full skeleton and rig, allowing her to be posed and animated.

You will also have had experience in placing, binding and manipulating joints in Maya, so this next section shouldn't prove too difficult as you will be repeating much of what you did when you constructed and connected her main skeleton previously.

1. Joint Limitations

As you can imagine, adding bones into the face to manipulate each key muscle could result in a heavy increase in your joint count. Giving your character the ability to show even the most subtle of emotions, while retaining an organic feel, can mean adding up to 100 new joints, which then need to be bound to the mesh, posed and eventually animated.

If your game is narrative heavy, with lots of cinematic cut scenes and close ups, then you will need a higher joint count or the facial movements will be stiff.

Again, most of the decisions concerning how many joints are available to use will come down to the target platform, the game engine and the actual game you are making. Less processing power will essentially mean less joints are available to you, so you may have to be creative.

It’s not all about joint numbers though, you also need to consider what attributes can exported when animated. Traditionally, animation applied to the rotation values was all that was exported and used in the game engine, with the exception of a few key joints used to plot the character's position. Adding in the translation and scale values at that time could prove expensive as it ultimately increased the amount of data which was exported and processed.

Luckily, these days this is no longer a major concern for most platforms, but it is always worth checking which attributes can be animated before you commit to your facial rig.

With this in mind, for Kila you will be using a basic facial setup using 26 joints. This will be in addition to the ones which already exist for the jaw and tongue, bringing the total up to around 30.

You don't have to worry too much about which attributes will be used, so feel free to proceed with both rotations and translations in mind. This should make your life much easier.

2. Joint Placement

Now you know the amount of joints available to use, and which attributes will be exported, it’s time to dive in and start to place the joints which will drive Kila's face. What you are looking for at this stage are the points that move the most when you want to express a certain emotion, or even just when you speak and form key words.

If you look at yourself in a mirror and pull a few faces (angry, happy, and so on) you will get a good idea of which muscles move the most.

Let’s first look at where you are going to place your 26 joints.

There is already a joint in place to open and close the jaw, but you will still need to pull and push the lips around to achieve more expression and expose her teeth, especially when talking. Because of this, you should allow 10 joints from your budget for this area.

A joint will be needed to help drive each eyelid, allowing her to blink, squint and look shocked.

The eyebrows also play a key part in showing emotion so they need to be independently raised, lowered, and also allow for other poses.

Now you'll have allocated 20 joints. Ten for the lips, four for the eyelids and six for the eyebrows. You have six joints left in your budget.

As you move your mouth it also affects your nose and nostril area, so it would be a good idea to allow a joint on either side of the nose. This will help her to sneer, and also make her movements more organic in this area.

As you open your mouth you will notice that your cheeks are pulled in slightly, but including joints here will have another benefit. Moving them in the opposite directions will also allow you to puff out her cheeks.

The final two joints will sit on her upper cheeks. Again, this will allow more fluidity in her movements and help enhance expressions like her smile and squint.

Now you have a good idea of where each joint is going to be used, let’s start to build them.

1. Focus on the left side first and go to Skeleton > Joint Tool.
2. Looking just from the front view initially, work your way around placing your joints. Make sure these are not parented to anything just yet. Leave each floating in space.
3. You may also want to include two central joints for the upper and lower lips.

When you are happy with how they look from the front, switch to the side view, (or perspective if you are happier working in that view) and move each joint either backwards or forwards until they lie on the surface of the model.

Next, position the joints which will be used for the eyelids in the exact same place as the existing eye joints. This is so once rotated, the geometry for the eyelids will pivot around the eyeball.

You may also want to move the upper eyelid joints up slightly, and the lower ones down a touch so they are easier to see and select.

Finally, rename these joints appropriately following the example image below, making sure that the joints which are on the left have a prefix of “L_”.

The joints are in place for the left side of Kila's face, but before you create the right side you have to parent these to the main skeleton.

Again, it’s not just a case of parenting them all to the head. This would be correct for the majority of the joints, but what would happen if the jaw opened? The lower lips would remain in place and stretch—not ideal.

Instead the lower lip joints should follow the jaw's rotation.

1. Select all the joints, except for the ones forming the lower lips.
2. Now, holding Shift, add the Head joint to the selection.
3. Pressing P will now parent these joints to the Head joint. (You can also go to Edit > Parent to perform the same action.)
4. Next select the lower lip joints.
5. Hold Shift to add the Jaw joint to the selection.
6. Again, press P to parent the lips to the jaw.

With the joints correctly added to the main skeletal structure you can now easily generate the joints needed for the right side of her face.

1. Go to Skeleton > Mirror Joint and open the options.
2. Set Mirror Across to YZ as this will mirror the selected joints from +X to –X, so from left to right.
3. Set Mirror Function to Behavior.

Reminder: The Mirror Function will dictate how each joint's orientation will be treated. Behavior will mirror the joints while also inverting their orientation, which in this case will keep X pointing down the bone, which is what you want. Orientation will retain the exact same rotational axes as the source joints.

Under Replacement Names for Duplicate Joint set Search For to “L_”, and Replace With to “R_”. This will automatically rename your left joints to the naming structure for right joints.

Now select each joint in turn and press Apply to generate the one opposite.

The head joints are now in place and ready to be connected to the mesh.

3. Connect to the Model

The new facial joints are added but at the moment they don't do anything. This is because they aren't yet physically connected to the model, so they aren't affecting any vertices.

1. Select each new joint in your facial setup.
2. Now add the model of Kila to the selection.
3. Go to Skin > Edit Smooth Skin > Add Influence.

If you move the facial joints you will see that the model will now deform with them. Yes, the deformation is most likely hideous, but now they are attached and influencing the mesh you can move on and begin to refine their influence.

4. Adjust Weighting Information

Each joint is now affecting a certain amount of vertices surrounding it, but the result isn't as smooth and organic as you might like.

The good news is that you can now go in and adjust this influence using the Paint Weights Tool.

Note: Adjusting the weighting on each joint is a process which you will come back to as you progress through the tutorial. Once you start to define facial poses, you will no doubt discover areas which need refining or adjusting, so don't worry if the weighting isn't perfect at this stage.

Select the Kila model and go to Skin > Edit Smooth Skin > Paint Skin Weights and open the options.

The Paint Skin Weights window will look a little intimidating, but don't worry, you will only be using a few key areas for now.

You will also notice that your model has turned black, with a greyscale falloff around the selected joint. What this is showing you is the current joint's influence on the model. The whiter the polygons, the more influence they have.

By working your way through each joint you can interactively paint the way each influences the model.

Let’s look at adjusting the weights on one joint, and then the same procedure can be applied to the rest.

1. It’s much easier to work on a joint’s weighting information if it has been moved or rotated, so for now temporarily rotate the L_EYELidUpper joint down along the X axis to around 45. Don't worry if there is little or no movement on the model at this stage.
2. Right click on the L_EYELidUpper joint and go to Select Influence in the marking menu.

You are now ready to start working, and posing the eye means you will see the weighting information update in real time as you’re adjusting it.

1. Back in the Paint Skin Weights window, now select Replace as your Paint Operation.
2. Next to Profile, select a softer brush.
3. Set Value to 1 and Opacity to around 0.3.

Note: Opacity will paint a percentage of Value and gradually build up the weighting for you until it reaches 1, which means the vertices are affected fully by the specific joint.

With those set you now need to paint over the vertices you want to be influenced by the L_EyeLidUpper joint.

Note: If you paint in too much influence simply set Value to 0 and gently remove the joint's effect.

As a final step, before you move on to the next joint, set Paint Operation to Smooth, with a small Opacity of around 0.25.

Clicking Flood will wash over the whole model, smoothing the weights for you on the selected joint, which can help to make its influence less rigid and more organic as it moves.

You can now repeat these steps for each of the left hand side joints in the face, remembering to pose it first before you adjust the weights so you can visualize its influence.

As mentioned earlier, this can be seen as a “first pass” at the weighting as it will be much easier to work on it later once you have some facial expressions defined.

Note: Don't forget to also rework the weighting on the head, jaw and tongue joints too as these will all be different now the new joints have been added.

Before you move on however you will need to copy the weighting information across to the right side of her face.

1. To do this simply select the Kila model and go to Skin > Edit Smooth Skin > Mirror Skin Weights and open the options.
2. Set Mirror Across to YZ, and also make sure Normalize is checked to clamp the weighting between the values of 0 and 1.
3. Now click Apply and you are done.

5. Build Your Rig

The joints now all deform the vertices of the face, and if needed you could move ahead and begin to animate these directly. As with any character though it’s always better to add a basic set of controls on top of these joints, much like you did previously with Kila's body, not only to make animation much easier but to also make the scene visually cleaner.

There are many options available to you when constructing a facial rig, and how you approach it could come down to the complexity of the face and what you need it to be able to do. The more joints the face holds, the more poses are possible, so adding icon-based controls and sliders may be seem daunting, but it will make your life easier in the long run.

What you will do here, and also in the following blend shape tutorial, is rely on a basic set of attributes which will drive the face, pushing it into specific poses and expressions. It may be that all the character needs to do is show a few key emotions for when they are happy, scared or damaged in game. If this is the case then you could have a handful of new attributes which simply activate these key expressions.

You could go further, as we will, and include actual mouth shapes allowing her to form words as well as smile, grimace and frown. This can then be extended to her eyes and eyebrows for even more control and emotion.

Your first step is to decide on the rig you want to create, and here you will simply add a series of attributes to an existing controller and then connect these to the facial joints using Set Driven Keys to drive their positions.

Note: Even though this is a basic setup, once created you can always add another layer, like the icon-based example above, to drive these attributes.

1. Select the HeadControls icon in the scene.
2. With this selected go to Modify > Add Attribute and open the options.

This tool will give you the ability to create new dummy attributes, which will sit on the HeadControls icon.

1. For the first attribute, add Jaw Open into the Long Name box.
2. Now set the Minimum value to 0.
3. Set the Maximum value to 10.
4. Set the Default to 0.
5. Finally click Add to create the new attribute.

Now that wasn't too taxing, and the result is a dummy Jaw Open attribute which is locked to values between 0 and 10. The problem is it doesn't do anything yet, but we will come to that later.

Not all your attributes will want to be set to just positive values. It may be that for the eyelids for example, a generic Eyelid attribute has values of 0 to 10 which widens the eyes, while 0 to -10 closes them. So on one slider you can open, close and also widen the eyes.

Your next step is to go through and generate all the dummy attributes you might need, but don’t worry if you miss some as you can always add more later.

1. Begin with Jaw Open, Jaw Slide and Jaw Twist. The last two attributes should range from -10 to 10 so they can move and rotate in both directions.
2. Next add your key expressions, so Happy, Shock and Angry for starters.

Now focus on the actual mouth shapes, and don't be afraid to test these yourself in a mirror to see which letters form similar shapes.

1. Add new attributes for O, CDENSZ, AKI, L, MPB, FV, Ooo, Ahh and Th.
2. Also include a Smile, Frown, Grit and Puff in this section.
3. By now you should be getting the general idea, but before you move on also include attributes to control the eyelids and eye brows too. These can all be seen in the image below.

6. Drive Your Poses

Your joints are positioned and bound to the mesh. You now also have a good set of dummy attributes just itching to drive those joints, so let’s make them happy and connect them.

To drive the poses you will be using Set Driven Keys. What these will do is treat the dummy attributes as mini time sliders, so as an example, at 0 the specific joints will be in the default position, but at 10 the joints will move into place. You can also scrub between 0 and 10 to gradually move the joints, rather than them snapping into the desired pose.

Let’s jump right in and see how this works.

First, go to Animate > Set Driven Key > Set… This will open the main Set Driven Key window.

Let’s look at the Jaw Open attribute first.

1. Select the HeadControls icon and click Load Driver in the Set Driven Key window. This will add the icon, and its keyable attributes into the upper section of the window.
2. Now select the Jaw joint and click Load Driven, adding this to the lower section of the window.

For this particular attribute you only need to focus on one rotational axis as the jaw will only need to open in one direction. Further control can be added through the Jaw Slide and Jaw Twist attributes.

1. Under the Driver section in the Set Driven Key window, select Jaw Open as the active attribute.
2. Beneath this, in the Driven section, select the Rotate Z attribute only.
3. Now press Key to store this pose.

That’s the first pose defined, so at 0 the jaws rotation is 0. Next you need to add in the final pose.

1. Select the HeadControls icon and set Jaw Open to 10.
2. Now rotate the Jaw joint to around -14.5, or wherever you are comfortable.
3. Press Key to store this new pose.

Now when you adjust the Jaw Open attribute, Kila's mouth will open and close. This is just driving one joint, but you can add many more to the same attribute. In this case you also want the cheeks to pull in slightly as she opens her mouth, as yours or mine naturally do.

1. Select the L_Cheek and R_Cheek joints and click Load Driven in the Set Driven Key window.
2. This time you only want to focus on their Translate X attributes, so select these now.
3. Make sure the Jaw Open attribute is back on 0 and press Key to store this pose.
4. Now set Jaw Open to 10 and move each cheek joint in slightly.
5. Press Key when you are happy to store this pose too.

It’s subtle, but when the mouth opens now, it looks a lot more natural. You can also go further if you like and adjust the lip joints to pull them in slightly at either side of the mouth.

The beauty of Set Driven Keys is that you don't just have to restrict the poses to either end of the attributes scale. If you wanted more control over each pose, or the movement towards that pose, you could add more positions in between the key ones.

As an example, setting Kila to stick out her tongue would result in the actual tongue model popping through her bottom lip once she was half way towards the key pose. This is because the joints are simply moving from A to B, from value 0 to 10.

Adding an extra pose for the tongue could delay its movement or even raise it slightly, ready to then come out once the mouth had begun to open. So in this instance the tongue would move from A to C, before finally reaching B, while moving through values 0, 5 and 10.

Rather than walk you through each and every pose, which could take a while and turn this into a tedious tutorial, what you should do now is repeat this process and generate your own poses to match the names of each attribute.

Remember to store each pose, and also include in between poses too if you need them. Experiment and don't be afraid to look in a mirror as you form the expressions to help guide you.

7. Editing Your Poses

Once you have defined your poses, reworking or editing them isn't a huge task and can be done outside of the Set Driven Key editor.

Being keyframe-based means you can adjust each attribute in the Graph Editor as you would an animated object in Maya.

• Select the Tongue_Base joint and go to Window > Animation Editors > Graph Editor.

What you will see are a series of curves which can be adjusted or even added to, allowing you to alter the way the joints move.

Adjusting the tangents can set the joints to ease in and out of their poses, while adding more key frames can change its trajectory. You could even use the Graph Editor to simply rework the translation or rotation on the selected joint, making the pose more subtle, or more expressive.

Conclusion

So that’s the joint-based approach to setting up facial animation. It was a brief overview but hopefully you can see the process taken to add poses to Kila and allow her to emote.

With this approach you will find that you are effectively pulling and pushing areas of the face around to form poses and mimic the movement of the facial muscles.

One of the good parts of using joints is that once they are built and set up, you can easily add more facial poses with very little hit to the data which is exported and handled by the engine. You’re not adding anything else to the rig, just moving the existing joints into new positions.

Another benefit to using joints is if the model changes for some reason, the poses are all stored in the joints. So once they are re-skinned they will adapt, making this ideal for level of detail models as the animation will also traverse down the LODs.

One drawback is that as the vertices are merely influenced by the joints, you may not get as smooth a result as you would like, especially on a lower polygon model.

In Part 8 we will take a look at another approach to facial animation, and see how to create, set up and work with blend shapes in Maya.