Evaluating Natural Language to Bash Command Translation on Consumer-Grade Hardware
Keywords: NL2CMD Translation, Bash CLI, Human-Computer Interaction
Abstract: Efficient and accurate translation of natural language to Bash commands (NL2CMD) can simplify user interactions with the Linux command line interface. Recent translation advancements have predominantly utilized large language models, which are either costly or impractical for deployment on consumer-grade hardware. We evaluate the efficacy of the StarCoder2-3b model, fine-tuned on a dataset with Linux manual page context, using execution-based evaluation. Our findings show that while our model does not surpass the accuracy of state-of-the-art models, it offers a significant improvement over previous models capable of running on similar, lower-specification systems. This research advances the understanding of NL2CMD translation on accessible hardware and sets the groundwork for future explorations into resource-efficient computational models for human-computer interaction.
Submission Number: 8
Loading