SimDiff: Simulator-constrained Diffusion Model for Physically Plausible Motion Generation

Anonymous

Prompt ① — "Person is adjusting their sitting position."

MDM (Original)

MDM + PhysDiff

MDM + SimDiff

Prompt ② — "A person does a backflip."

MDM (Original)

MDM + PhysDiff

MDM + SimDiff

Prompt ③ — "A person gets down on the floor and crawls forward."

MDM (Original)

MDM + PhysDiff

MDM + SimDiff

✦ Gravity Variation (wind = 0)

Prompt: "A person runs forward steadily."

⬇
Gravity
1.23
↔
Wind X
0
↕
Wind Y
0

Low Gravity (Moon-like)

⬇
Gravity
9.81
↔
Wind X
0
↕
Wind Y
0

Earth Gravity

⬇
Gravity
24.8
↔
Wind X
0
↕
Wind Y
0

High Gravity

✦ Planar Wind Variation (g = −9.81)

Prompt: "A person steps forwards carefully."

⬇
Gravity
9.81
↔
Wind X
0
↕
Wind Y
0

No Wind

⬇
Gravity
9.81
←
Wind X
-1.0
↕
Wind Y
0

Wind Left

⬇
Gravity
9.81
↔
Wind X
0
↓
Wind Y
-1.0

Wind Forward

✦ Compositional Generalisation to Unseen Environment

Prompt: "A person jumps high from a standing position."

⬇
Gravity
9.81
↔
Wind X
0
↕
Wind Y
0

Earth + No Wind

⬇
Gravity
1.62
↔
Wind X
0
↕
Wind Y
0

Moon + No Wind

⬇
Gravity
1.62
→
Wind X
2.0
↕
Wind Y
0

Moon + Wind Right

⬇
Gravity
1.62
↔
Wind X
0
↑
Wind Y
2.0

Moon + Wind Backward

⬇
Gravity
1.62
→
Wind X
2.0
↑
Wind Y
2.0

Moon + Wind Diagonal