Collaborative three-stream transformers for video captioning

Hao Wang, Libo Zhang, Heng Fan, Tiejian Luo

Abstract: Highlights•COST is proposed as a novel multi-branch framework for video captioning.•Designed module aligns interactions across branches, leading to precise captioning.•Proposed training objective enhances COST via constraints on embeddings’ semantics.•Abundant experiments show our method performs favorably against the state-of-the-art.