DirectXGraphics: Vertex Blending
Author: Jack Hoxley
Written: 31st July 2002
Contact: [EMail]
Download: Vertex Skinning.zip (58kb)

Contents of this lesson
1. What is Vertex Blending?
2. How to set up simple vertex blending
3. Extending this to use all 4 matrices
4. Taking it all the way: Indexed vertex blending

1. What is Vertex Blending?

I'm going to have to assume that by this point in the series you have a reasonable understanding of the Direct3D pipeline - if not, this tutorial won't make a huge amount of sense to you.

As you should be aware, when you send vertices to the device for rendering (Device.Draw* calls) they go through a number of stages before final rasterization. In particular, there is the transformation engine - this is near to the start of the process, and deals with multiplying every vertex in the data stream by the world matrix (view and projection can be done later). The world matrix will typically rotate scale and translate a given object.

As it currently stands, the world matrix transforms every vertex in exactly the same way - the whole mesh is rotated, or the whole mesh is translated across world space. Using current techniques you cannot apply different transformations to different parts of the same mesh without rendering each part separately. This works well in most cases, but it's lacking a more fluid, dynamic and artistic touch.

Take a look at the following screenshot (from the sample):

Nothing hugely exciting about this - a simple textured octagonal prism. Given current methods, and without altering the raw-geometry how would you make the above mesh render like this:

Interesting eh? Even if you were allowed to alter the actual geometry being rendered, the above shape would still be a little tricky to create (unless you used a software emulation of the D3D technique!).

To sum up, vertex blending allows us to specify up to 4 matrices per rendered mesh, and to specify how the transformations are distributed across the geometry. The above image was generated using 2 matrices, with the second matrix focusing mostly on the bottom of the prism (hence why most of the distortion is visible there).

Using vertex blending with bones is a very common technique in commercial games. Take a humanoid mesh, and set up a simple skeleton (you may need to read up further on skeletal animation), typically a set of vertices are attached to each node in the skeleton. As the skeleton is animated it moves the (rendered) mesh around. Using vertex blending you can let any vertex be influenced by more than one matrix (where each matrix represents a bone for example). This creates much more realistic character animation - much smoother (you can even make muscles flex) and fluid.

2. How to set up simple vertex blending

Vertex blending is not a hard effect to master, the most difficult aspect is deciding how you are going to use it. You will need to determine a pattern or algorithm to set blending values for each vertex, and then to decide what transformation to store in each slot.

The first step is to check for hardware support of vertex blending matrices. You can only ever apply a maximum of 4 matrices per vertex whatever the hardware.

The next step is to alter the vertex format - we must specify a blending 'weight' for each vertex you want to render. This is a floating point value indicating how much each matrix is going to contribute to the overall result. Typically this will be left in the range of 0.0 to 1.0, with 0.0 being disabled/no effect, and 1.0 being fully effective. 1.0 doesn't mean that it will be the only matrix that affects the current vertex, it just means that the current matrix is not scaled. The general formula being:

P_final = P_vertex * W₁*M₁ * W₂*M₂ * W₃*M₃ * (1.0-W₁-W₂-W₃)*M₄

where P_final being the final world-space coordinate, P_vertex being the original model-space vertex coordinate. W₁, W₂ and W₃ are the three floating-point blending weights. M₁, M₂, M₃ and M₄ are the four matrix slots.

The vertex data type, for this simple sample, is altered in two ways: The actual data structure and the the FVF descriptor.

This sample only uses the first two matrix slots (of the four available), that is the D3DTS_WORLD and D3DTS_WORLD1 slots defined in the Direct3DDevice8.SetTransform(  ) function. If we plug this into the general equation, we can now see the following:

P_final = P_vertex * B1*D3DTS_WORLD * (1.0-B1)*D3DTS_WORLD1

that is, as you might recognize, the standard linear interpolation equation. As B1 tends towards 1.0 the vertex will be more affected by the D3DTS_WORLD matrix, as it tends towards 0.0 it will be affected by the D3DTS_WORLD1 matrix. Anything in the middle will be a mix of both!

One slight problem remains. When we load geometry from a .x file, it probably won't know that it's going to be used for vertex blending purposes - and as such, won't have the necessary blending weight parameter in the vertex structure. We must therefore add this member to the mesh's vertex format. When you've loaded the mesh as per normal, we use this following short piece of code to alter the vertex format:

CloneMeshFVF( ) will have now allocated space for the blending weight, BUT it won't have put any data in - it will be left as 0. Therefore the next step is the all important - setting blending weights:

By this point we've done all the initialization necessary to use vertex blending in our application. There is one final render state to observe before rendering the mesh:

That is all of the vertex blending samples at the simple level. As it currently stands, whatever type of matrices are defined in D3DTS_WORLD and D3DTS_WORLD1 will affect all rendered geometry. The sample code makes use of rotating through two axis (Y and Z) to get the results shown in original screenshots. The code for this is not very important, and you can find it in the downloadable archive.

3. Extending this to use all 4 matrices

The previous section dealt with using only the first two matrix slots. Just to cover all the bases I'm going to give a quick demonstration of how you could use all 4 matrices - with these two examples you should be able to work out how to do 3-matrix blending. The downloadable sample code does not include any of the source from this point onwards, but you should see how it will easily fit in...

For using all 4 matrices we must change the FVF and the vertex declarator again:

When you are ready to render the geometry you must use the following two lines of code:

4. Taking it all the way: Indexed vertex blending

Vertex blending has been available through Direct3D for several versions now, the methods discussed thus far are nothing new. However, there is one new addition to vertex blending for Direct3D8 - indexed vertex blending.

This still limits you to a maximum of 4 matrices per vertex, BUT it allows you to specify up to 256 matrices for each call. How does this work? well, it's actually very clever...

If you're an experienced DirectX programmer, you may well remember using/learning how 8-bit palletized textures work. Each pixel stores an 8bit number - this indexes to to a palette of 256 different colors, but each color in the palette is defined as a 24 bit RGB triplet. A similar idea applies here.

For each vertex we still have the blending weights (1,2 or 3 of them), but we also add a 4-byte packed index value. This 32bit long is 4 indices packed into one variable. The idea being that you can store the 3 weights as appropriate, but then use the indices to select which 4 matrices they apply to - from a list of up to 256.

Say for example you had a list of 35 matrices setup, and for a triangle of 3 vertices you wanted 11 different matrices (based on each vertices distance from a bone node for example). This wouldn't be possible using the traditional method.

For example:
V0 = matrices 1, 3, 5, 7 weights of 0.2, 0.3, 0.4
V1 = matrices 1, 3, 6, 9 weights of 0.2, 0.6, 0.1
V2 = matrices 2, 4, 6, 8 weights of 0.1, 0.05, 0.3

to use indexed vertex blending we must (again) alter the vertex structure and the FVF:

Before you actually use indexed vertex blending you need to check hardware support. Only the more recent hardware will support any significant number of matrices - my ATI Radeon8500 supports 58 matrices (slots 0 to 57), and not the full 256 possible.

first: create and apply each matrix. You can't apply more matrices to the device than specified in the above enumeration. The following code will allow you to set a matrix in slot "I":

Configuring Direct3D to let you use indexed vertex blending is not a complicated task - however, writing an algorithm that selects the correct weights and matrix indices is going to be the far harder challenge. It may well be worth writing a plugin/tool for a 3D renderer that allows an artist to specify weighting values.

One solution I have thought of - assuming you were using this technique in conjunction with skeletal animation (I would!):
assume a reasonable skeleton with 58 bones (the same number of matrices my Radeon8500 supports). We can generate a matrix for each bone node (as a trivial example - shoulder->elbow->wrist nodes) quite easily. Then, we loop through every vertex in the mesh/skin, and calculate a list of distances from the vertex to each bone-node. Sort this list so we can pop-out the 4 closest bones (this isn't a very smart method, but it would work). We can then work out as a proportional value how close each bone is to the vertex and generate 3 blending weights. Algorithm complete.

In using a solution like that the artist need only specify a skeleton animation and a mesh-skin, and the resulting animation would be a very convincing and fluid system.

So, that is this tutorial completed. Uses for vertex blending can be huge - but I'll have to let you do your own research there.

You can (and I strongly advise you do) download the sample code from the top of the page, or from the DirectX8 Downloads page.

Enjoy!