NeurIPS#4644

1. Text-to-Motion Generation

1.1 Additional Comparison with Baseline Text-to-Motion Generation Methods

We show additional result comparisons between our method and four baseline methods: MDM, MotionLCM-V2, MLD++, and MARDM. Our method generates motion that is more realistic and more accurately follows the fine details of the textual condition.

A man steps to the right steps in a small counterclockwise circle, throws right arm then steps in a larger backward counterclockwise circle.

Ours

MDM

MotionLCM-V2

MLD++

MARDM

A person stands on one legs in yoga pose.

Ours

MDM

MotionLCM-V2

MLD++

MARDM

A person climbs a ladder.

Ours

MDM

MotionLCM-V2

MLD++

MARDM

Person turns to left, takes three steps forward, sits down, and then walks back to starting place.

Ours

MDM

MotionLCM-V2

MLD++

MARDM

A person walks forward, stepping up with their right leg and down with their left, then turns to their left and walks, then turns to their left and starts stepping up.

Ours

MDM

MotionLCM-V2

MLD++

MARDM

1.2 Visualization Gallery

Our method is capable of generating high-quality, textual instruction-following 3D human motions.
We include additional 9 distinct motion examples generated by our method.

The person puts something on its side and then brings it back to normal.

A person walks forward slowly, their arms swinging slightly, then they turn around.

A person stands still and the takes one quick step forward.

An individual takes a long slow drink of something.

A person jumps in the air, then abruptly stumbles to his left as if he had been pushed, and finally he regains his balance.

A person runs to their right, then left, then right again, and finally walk abck to their starting position.

A person is bent over forward and moves their body left to right like a snake several times.

A person is sitting down, using a phone with their hands and puts it up to their ear.

A person who is standing with his hands by his sides turns to the left as he takes four steps and stops.

2. Text Driven Controllable Motion Generation

2.1 Comparison with Baseline Text Driven Controllable Motion Generation Methods

We show result comparisons between our method and two baseline methods: OmniControl and MotionLCM-V2 with ControlNet.
Our method generates motion much faster (2.51 second) and near-flawlessly follows the user-provided controling signal.

A person slowly walks in an s shape while shifting weight between each leg.

Pelvis

Ours+ControlNet

MotionLCM-V2+ControlNet

OmniControl with Classifier Guidance

Left Foot

Ours+ControlNet

MotionLCM-V2+ControlNet

OmniControl with Classifier Guidance

Right Foot

Ours+ControlNet

MotionLCM-V2+ControlNet

OmniControl with Classifier Guidance

Head

Ours+ControlNet

MotionLCM-V2+ControlNet

OmniControl with Classifier Guidance

Left Wrist

Ours+ControlNet

MotionLCM-V2+ControlNet

OmniControl with Classifier Guidance

Right Wrist

Ours+ControlNet

MotionLCM-V2+ControlNet

OmniControl with Classifier Guidance

2.2 Visualization Gallery

Our method is capable of generating high-quality 3D human motions following textual instruction and control signals.
We include additional 12 distinct motion examples generated by our method (2 for each controlled joint).

A person slowly walks forward with arms swinging, turns in a clockwise circle.

Person walks quickly down a short incline.

A person takes eight steps forming a complete circle.

A person shifts rightwards and then shifts back.

The toon is walking acroos the plane at a diagonal pattern, reaching the end of the plane & turning around.

A person walks in a curve to the left.

This person stumbles right and left while moving forward.

A person carefully stepping backwards.

A person picks something up with each hand and then stacks the item from their left hand on top of the item their right hand.

A person jogging in place.

The person is moving from side to side.

A person jumped on the place.

2.3 Spatial Editing

Our method is capable of spatially editing 3D human motions. We include additional 2 motion examples generated by our method.

Original: A person bends to collect something, turns and goes back.
Editing Upper Body: A person holds his hands up.

Original: A person is sitting on a chair.
Editing Lower Body: A person performs a kick.

3. Direct Text-to-SMPL-H Mesh Vertices Motion Generation

3.1 Benefit of Direct Text-to-SMPL-H Mesh Vertices Motion Generation

We show additional result comparisons between our direct SMPL-H mesh generation method and generated-joints-to-SMPL-H meshes through a SMPL fitting model approach.
Our direct SMPL-H mesh generation method produces more realistic SMPL-H mesh vertices motions and better captures natural human movement.

A person is sitting.

Better Motion, No Self-Penatration,
Better Hands Movement,
Implicitly Modeled DMPLs (See Belly Flesh Movements).

Direct SMPL-H Mesh Vertices Generation

Worse Motion, Self-Penatration,
Unnatural Hands Movement,
Does Not Model DMPLs

Joints->Mesh Vertices

A person walks backwards.

Better Motion, No Jittering Head,
More Natural Hand Movements.

Direct SMPL-H Mesh Vertices Generation

Worse Motion, Jittering Head
Unnatural Hand Movements.

Joints->Mesh Vertices

3.2 Visualization Gallery

Our method is capable of directly generating high-quality, textual instruction-following SMPL-H mesh vertices motions.
We include additional 9 distinct motion examples generated by our method.

Man extends right arm direcly in front of him, moves it in front of his body from left to right and back down

A person steps forward and leans over; they grad a cup with their left hand and empty it before putting it down and stepping back to their original position.

A person who is standing with his hands at this sides reaches down to his right, picks up something, moves to his left and places it down and retun to his standing position.