Generating motion with AI Video Mocap in iClone

iClone Video Mocap adopts an AI-powered advanced video-to-motion generation process that cuts down on production time and costs by bypassing the use traditional mocap devices. The following diagram illustrates how to generate motion from a video and convert it to Reallusion motions applicable for humanoid iC/CC characters.

iClone Video Mocap is able to detect the target actor in the video with some proper guidance from the user. The Video Mocap detects the target actor and its full body part in the video for motion capture. Follow the instruction below to capture motion from any clear actor in your video.

Product page: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/welcome.htm

Online Manual: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/welcome.htm

3D Textures – PBR Shader Type Maps

PBR (Physically-Based Rendering) uses consistent lighting behavior to make materials look realistic in any environment. Stick to metal-rough or spec-gloss workflows and make sure your maps are calibrated properly (e.g., roughness should be in linear space, not gamma corrected).

Each material has 9 map channels that collaborate for materials of the PBR Shader Type. Open Texture Settings section in the Modify > Material tab to see these channels.

Base Color map

• The most frequently used texture mapping method. It wraps the bitmap image onto the 3D geometry surface while displaying its original pixel color.
• Any bitmap image, such as scanned images or images captured by digital camera, can be used as diffuse map to represent photo-realistm.
• Users can also use image software to make pre-rendered texture effects such as shadow, bevel, bump, lighting or weathering effects.  This approach can effectively simulate real-world effects while greatly saving system resources and rendering time.

Bump map

• When you import an image to the Bump channel, a dialog will request for the image format. Choose the Bump option if the image you want to import is in a grayscale mode.

Normal map

A Normal map is made from a high polygon model. Its color representation will affect surfaces like a regular bump map while providing higher degrees of detail.

AO Map (Ambient Occlusion)

This texture mapping method enhance the concavity and convexity of an object by appending additional shading around the seams or overlapping edges.

Metallic Map

Metallic maps can determin the reflectivity of a surface ranging from completely insulate to metallic. White parts of the map stand for reflectivity while black parts of the map stand for diffusion.

Roughness Map

Roughness map is used to describe the coarseness from the definition of the microsurfaces. The surfaces mapped with white are relatively rough while the ones mapped with black are perfectly smooth.

The combinations of the Metallic and Roughness can thus depict almost every material in the real world.

Displacement Map

You can create monochrome image maps with popular image-editing tools such as Photoshop. By using grayscale height maps, you can push and pull the surface of a 3D model to achieve the ideal look you desire.

Using a vector-based displacement map can produce undercut details on your model. Unlike the height displacement map (monochrome) simply stretches out geometry, vector displacement produces refined curves and edges, creating an unparalleled sense of detail to your model.

Glow map

• This texture mapping technique allows users to control the glow shape, color and strength.
• Glow maps will blend with your original diffuse maps, so the lighter the diffuse color (or glow color), the stronger the glow effect.
• A bright diffuse map in combination with a bright glow map might cause overexposure.

Blend Map

You may add any kind of images as the Blend map to blend with the Diffuse map, which increases the details and subtleties of the object. iClone also provides three methods, Multiply, Addition and Overlay, for blending.

How to fix foot contact for custom imported animation

Sometimes while importing custom motion from another platform, the character in iClone viewport is placed above the ground level, it is easy to fix it, maybe try the following steps:

1. Select the character and press hotkey W, you will send the gizmo (the circle 1 in below screenshot), just drag the character down a little bit.

2. Check the ‘Foot Contact’ in Modify panel (see the circle 2). You will see now the character feet will be focused on the ground level.

3. Bonus: You may also double click the footprint in the viewport, or go to Scene Manager to select it, then press hotkey W, you can even move up or down to adjust the position of the footprint as well.

Step-by-Step Guide: Integrating NVIDIA Omniverse with iClone 8 / Character Creator 4

Step 1: Setup

• Install and launch NVIDIA Omniverse Create (including Nucleus server)
• Install the latest iClone 8 and Character Creator 4
• Install the Omniverse Live Link plugin in iClone

Step 2: Create and Share USD File

• Create a new USD scene in Omniverse Create, saving it on the Nucleus server path
• Copy the USD file’s URL/link

Step 3: Connect iClone to Omniverse

• Open your character project in iClone
• Launch Omniverse Live Link, paste the USD URL
• Enable synchronization for character, animation, and scene

Step 4: Real-time Collaboration and Editing

• Edit character animation in iClone; changes update instantly in Omniverse
• Adjust lighting and materials in Omniverse; updates reflect in iClone
• Optionally add facial capture and physics simulations for enhanced realism

Step 5: Finalize and Export

• Save and merge all changes in Omniverse
• Render high-quality animations using Omniverse Renderer
• Or export FBX/USD files for further use in game engines

Tips

• Ensure USD files are stored on the Omniverse Nucleus server
• Keep FPS settings consistent to avoid animation sync issues
• Use Omniverse Material Builder to optimize material quality

This workflow significantly boosts multi-team collaboration efficiency by enabling real-time synchronized editing of characters and scenes, ideal for game development, film production, and metaverse projects.