Using Reinforcement Learning LoRA For Interactive Machine Translation Systems
Abstract: The use of large language models (LLMs) is growing due to their impressive performance on a wide range of tasks. As new versions of these models appear to achieve better results, their size often increases, making it more challenging to maintain different versions specialized in specific domains. However, by employing the Low-Rank Adaptation (LoRA) method, we can bypass this space limitation, as the fine-tuning changes of the model are stored in a file of just a few megabytes. In the Machine Translation (MT) field, it is common to have models specialized for particular domains or language pairs. In our case, we apply these models within Interactive Machine Translation (IMT), where it is crucial that the model generates high-quality translations and adapts to user modifications. We have incorporated Reinforcement Learning (RL) techniques to optimize the model using various metrics to enhance this adaptability further. Our results demonstrate that these methods effectively improve the quality of translations generated by the models, though in some cases, this comes at the cost of a slight reduction in generalization capability.
Paper Type: Long
Research Area: Machine Translation
Research Area Keywords: fine-tuning, interactive MT, reinforcement learning
Contribution Types: Model analysis & interpretability
Languages Studied: english, spanish, french, basque, swahili, lingala, yoruba
Submission Number: 1235
Loading