Failure Cases: Tracker-FG


Both Baseline and Our Metric Fail

Example 1

Human:
Video 1 | Video 2

VB-SC:
Video 1 | Video 2
Score 1 : 0.6052 | Score 2 : 0.8535

Tracker-FG (Ours):
Video 1 | Video 2
Score 1 : 0.9712 | Score 2 : 0.9905

The unnaturally shaped human hand remains stable and fails the Tracker-FG metric in detecting inconstencies in realism.


Example 2

Human:
Video 1 | Video 2

VB-SC:
Video 1 | Video 2
Score 1 : 0.8708 | Score 2 : 0.8981

Tracker-FG (Ours):
Video 1 | Video 2
Score 1 : 0.9914 | Score 2 : 0.9924

Very gradual variations in subject shape remains undetected in Tracker-FG as it looks for smoothness of tracks.


Example 3

Human:
Video 1 | Video 2

VB-SC:
Video 1 | Video 2
Score 1 : 0.7460 | Score 2 : 0.9394

Tracker-FG (Ours):
Video 1 | Video 2
Score 1 : 0.9932 | Score 2 : 0.9938

Similar case as Example 2.


Our Metric Fails but Baseline Works

Example 4

Human:
Video 1 | Video 2

VB-SC:
Video 1 | Video 2
Score 1 : 0.9818 | Score 2 : 0.9174

Tracker-FG (Ours):
Video 1 | Video 2
Score 1 : 0.9928 | Score 2 : 0.9947

VB-SC benefits from very limited camera motion in Video 1. For Tracker-FG, the primary subject has very little details making it harder to track points. Smooth regions in objects are less preferred.


Example 5

Human:
Video 1 | Video 2

VB-SC:
Video 1 | Video 2
Score 1 : 0.9452 | Score 2 : 0.9315

Tracker-FG (Ours):
Video 1 | Video 2
Score 1 : 0.9925 | Score 2 : 0.9938

The subject motion in Video 2 looks smooth, but its unnatural.