# Challenger: Affordable Adversarial Driving Video Generation

### Table of Contents

- Introductory Video
- Abstract
- Video Gallery 1
- Video Gallery 2

## A short introduction to Challenger.

## Abstract

Generating photorealistic driving videos has seen significant progress recently, but current methods largely focus on ordinary, non-adversarial scenarios. Meanwhile, efforts to generate adversarial driving scenarios often operate on abstract trajectory or BEV representations, falling short of delivering realistic sensor data that can truly stress-test autonomous driving (AD) systems. In this work, we introduce Challenger, a framework that produces physically plausible yet photorealistic adversarial driving videos. Generating such videos poses a fundamental challenge: it requires jointly optimizing over the space of traffic interactions and high-fidelity sensor observations. Challenger makes this affordable through two techniques: (1) a physics-aware multi-round trajectory refinement process that narrows down candidate adversarial maneuvers, and (2) a tailored trajectory scoring function that encourages realistic yet adversarial behavior while maintaining compatibility with downstream video synthesis. As tested on the nuScenes dataset, Challenger generates a diverse range of aggressive driving scenarios—including cut-ins, sudden lane changes, tailgating, and blind spot intrusions—and renders them into multiview photorealistic videos. Extensive evaluations show that these scenarios significantly increase the collision rate of state-of-the-art end-to-end AD models (UniAD, VAD, SparseDrive, and DiffusionDrive), and importantly, adversarial behaviors discovered for one model often transfer to others. Our code, models, and dataset will be released.

## Video Gallery

#### Examples from the Adv-nuSc Dataset

Due to file size limitations, videos are compressed and quality might be affected.

Realistic adversarial driving scene.

Overtaking from the left lane

Realistic adversarial driving scene.

Short-distance following

Realistic adversarial driving scene.

Approaching from the left lane

Realistic adversarial driving scene.

Frontal encounter, U-turn, and surpassing

Realistic adversarial driving scene.

Approaching from behind

Realistic adversarial driving scene.

Traversing in front of the ego vehicle

Realistic adversarial driving scene.

Zooming past from the left lane

Realistic adversarial driving scene.

Yielding, tailgating, and overtaking

Realistic adversarial driving scene.

Coming from the side and attempting to overtake

Realistic adversarial driving scene.

Attempted cut-in

Realistic adversarial driving scene.

U-turn at an intersection

Realistic adversarial driving scene.

Starting from roadside and surpassing from the left lane

Realistic adversarial driving scene.

Abrupt start from a parking lot

Realistic adversarial driving scene.

Lane weaving in a traffic jam

Realistic adversarial driving scene.

Tailgating and attempted overtaking

Realistic adversarial driving scene.

Cut-in from the left lane

Realistic adversarial driving scene.

Cut-in from the left lane

Realistic adversarial driving scene.

Starting from roadside and surpassing from the left lane

Realistic adversarial driving scene.

Frontal encounter, U-turn and tailgating in the night

## Video Gallery 2

#### Examples with multiple adversarial vehicles

Due to file size limitations, videos are compressed and quality might be affected.

Realistic adversarial driving scene.

Approaching from the left

Realistic adversarial driving scene.

Complicated maneuvers

Realistic adversarial driving scene.

Surpassing from the left lane

Realistic adversarial driving scene.

Approaching from behind and from right

Realistic adversarial driving scene.

Approaching from left and right

Realistic adversarial driving scene.

Complicated maneuvers

Realistic adversarial driving scene.

Leapfroging

Realistic adversarial driving scene.

Complicated maneuvers

Realistic adversarial driving scene.

Zooming past from the right and brake

Realistic adversarial driving scene.

Zooming past and cut-in

Realistic adversarial driving scene.

Lane blocking