Bridging Natural Language and MAVLink: Dataset Generation and SLM Fine-Tuning for UAV Command Execution

ACL ARR 2025 July Submission455 Authors

Abstract: Accurately interpreting natural language commands is crucial for deploying autonomous unmanned aerial vehicles (UAVs) in industrial environments. This study introduces the UAVIntent dataset by systematically using 16 Myers-Briggs Type Indicator (MBTI) personality types and drone operator roles for synthesizing the dataset with One-Shot Chain-of-Thought (CoT) based dataset pipeline. The dataset consists of 122 distinct command types derived from MAVLink documentation, totaling 19,088 data points. We conducted extensive experiments on this dataset, evaluating different approaches for converting natural language instructions into MAVLink-based commands and extraction of parameters by fine-tuning multiple small language models (SLMs) and a retrieval-augmented generation (RAG) framework leveraging Phi-3. Among SLMs, DistilBERT achieves the highest command classification accuracy (99.22%), outperforming BART-Base (97.65%), BART-Large (98.83%) and RAG + Phi-3 (97.42%). For parameter extraction, RAG + Phi-3 attains the highest exact match accuracy (90.74%) and slot-wise accuracy (95.47%), but at a significantly higher computational cost. DistilBERT, while less accurate (82.34% exact match, 92.35% slot-wise), offers a more time-efficient alternative for real-time UAV command processing.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: MAVLink, NLP, Drones, RAG, Slot Extraction, SLM Fine-Tuning, UAV Command Classification

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Reassignment Request Area Chair: This is not a resubmission

Reassignment Request Reviewers: This is not a resubmission

A1 Limitations Section: This paper has a limitations section.

A2 Potential Risks: N/A

B Use Or Create Scientific Artifacts: Yes

B1 Cite Creators Of Artifacts: Yes

B1 Elaboration: The artifacts used are mentioned clearly in our work

B2 Discuss The License For Artifacts: N/A

B3 Artifact Use Consistent With Intended Use: N/A

B4 Data Contains Personally Identifying Info Or Offensive Content: N/A

B5 Documentation Of Artifacts: N/A

B6 Statistics For Data: Yes

B6 Elaboration: section 3.2

C Computational Experiments: Yes

C1 Model Size And Budget: Yes

C1 Elaboration: section 4

C2 Experimental Setup And Hyperparameters: Yes

C2 Elaboration: section 4

C3 Descriptive Statistics: Yes

C3 Elaboration: section 4.3

C4 Parameters For Packages: N/A

D Human Subjects Including Annotators: Yes

D1 Instructions Given To Participants: Yes

D1 Elaboration: section 3.1

D2 Recruitment And Payment: N/A

D3 Data Consent: N/A

D4 Ethics Review Board Approval: N/A

D5 Characteristics Of Annotators: N/A

E Ai Assistants In Research Or Writing: Yes

E1 Information About Use Of Ai Assistants: Yes

E1 Elaboration: Section 3.1

Author Submission Checklist: yes

Submission Number: 455