ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning

Jelle Luijkx, Zlatan Ajanović, Laura Ferranti, Jens Kober

Abstract: Human teaching effort is a significant bottleneck for the broader applicability of interactive imitation learning. To reduce the number of required queries, existing methods employ active learning to query the human teacher only in uncertain, risky, or novel situations. However, during these queries, the novice’s planned actions are not utilized despite containing valuable information, such as the novice’s capabilities, as well as corresponding uncertainty levels. To this end, we allow the novice to say: “I plan to do this, but I am uncertain.” We introduce the Active Skill-level Data Aggregation (ASkDAgger) framework, which leverages teacher feedback on the novice plan in three key ways: (1) S-Aware Gating (SAG): Adjusts the gating threshold to track sensitivity, specificity, or a minimum success rate; (2) Foresight Interactive Experience Replay (FIER), which recasts valid and relabeled novice action plans into demonstrations; and (3) Prioritized Interactive Experience Replay (PIER), which prioritizes replay based on uncertainty, novice success, and demonstration age. Together, these components balance query frequency with failure incidence, reduce the number of required demonstration annotations, improve generalization, and speed up adaptation to changing domains. We validate the effectiveness of ASkDAgger through language-conditioned manipulation tasks in both simulation and real-world environments. Code, data, and videos are available at https://askdagger.github.io.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: We have changed the title and method name after discussing this with the action editor. For the rest, we have only made minor changes, such as correcting typos.

Video: https://youtu.be/kTahi8ShAAY

Code: https://github.com/askdagger

Assigned Action Editor: ~Tim_Genewein1

Submission Number: 4616