Featured image of post Netrunner music video: Weight painting for Metahuman Skeletons

Netrunner music video: Weight painting for Metahuman Skeletons

How to solve the challenges with weight painting against the metahuman body and face skeleton for Activ8te's latest track: Netrunner.

For Activ8te’s latest track, Netrunner, I am producing a music video that tells the story of Lucy, a netrunner who uses social engineering to extract secrets from a Militech corporate hacker. The music video is produced completely in Unreal and makes use of Unreal’s metahuman technology. Lucy is wearing a suit that has a collar going up to her neck. To make the suit follow the motions of the body, a technique called weight painting is used. For each bone in the skeleton, the suit needs to be painted in the areas indicating how much it should follow the movements of each bone. I commissioned the Lucy hero character on Fivver and only discovered later that the weight painting for the suit was not working when Lucy moved her head and neck.

The suit depears in the neck. Showing the weight painting for the clavicle.

Importing a Metahuman into Houdini

Houdini is likely the best 3D software ever written. It’s my goto software for anything 3D related. The benefit of Houdini is that it’s node based, data driven and I can write C like code against the data when needed. Here is how the data shows up in Houdini when importing the metahuman base skeleton and deforming it using the dance animation from the chorus; see previous post.

Lucy’s body skeleton deforms the suit. Nodes in Hodini to show the suit on top of the body.

This looks fine but reveals the underlying problem. The suit was painted only against the metahuman body skeleton. Metahumans come in two parts: the body and the face. So, let’s try to bring in the face as well.

The face skeleton does not know how to follow the body skeleton. Nodes for naively attaching the face to the body.

Attaching Metahuman Face to Body with a bad hack

While there surely must be a way to do this properly, I figured I would just let Houdini automatically weight paint the face so that it follows the body. This was a little bit of an adventure.

Biharmonic capture allows Houdini to compute weights automatically. Nodes to automatically capture the weights.

The face now follows the body but it does not quite fit.

Fortunately, Houdini has a great SOP to deform one mesh followig the deformations of another mesh. This allowed me to conform the border of the face mesh with the border of the body mesh.

Using PointDeform to match the border of the face mesh with the body. Simple set up for Houdini’s PointDeform node.

The problem with PointDeform is that it requires the meshes to be close to each other. The further away the deformed mesh is from the mesh that guides the deformation, the less sense the deformations make. Fortunately, there is a way in which this can be combined with another SOP called BlendShapes.

Paining a blend mask where BlendShapes should take over from PointDeform. The nodes that combined BlendShapes and PointDeform to integrate the face with the metahuman body.

Mixing PointDeform and BlendShapes allows us to follow the motion.

Using the Metahuman Hack to Paint Weights

With the prep work out of the way, it’s time to look at the suit mesh and see how it follows the overall body. As we can see with the applied animation, the suit is not properly following the whole body. There clearly is a problem with the painted weights.

The suit does not follow the motion of the whole body. Painting the weights for the neck which were completely missing before.

After applying additional weights and fixing some other weights that were not painted correctly, the setup looks better.

Fixing the weight painting leads to proper deformation of the body suit.

Now, we can export the asset as FBX from Houdini and reimport it into Unreal Engine for a final check. As it turns out, my hack did not entirely follow the real metahuman deformations, so it required a few round trips to get to the proper results. Apparently, Unreal 5.3 has built in tools for weight painting now which should make this whole process much easier.

Stay tuned for the release of Activ8te’s Netrunner track and the publication of the Cyberpunk inspired music video. Follow Activ8te on Bandcamp or any of the streaming services.

This was just another of the many problems, I encountered when producing the music video for Activ8te’s new Netrunner track. You can learn more about EDM, by reading my guide on electronic dance music.

The views expressed on these pages are my own and do not represent the views of anyone else.
Built with Hugo - Theme Stack designed by Jimmy