Fine-Tuning Large Language Models for Multitasking in Online Shopping Using Synthetic Data
Keywords: Large Language Models, KDD Cup, Qwen, AWQ, Fine-tuning, E-commerce, Multitask
TL;DR: Innova team submission for Amazon KDD Cup 2024 Challenge for LLMs - 4th position in the Multilingual Track
Abstract: This paper presents an approach by the Innova-team to the KDD Cup 2024 ShopBench challenge, specifically detailing the 4th-place solution in Track 4: Multi-lingual abilities$\footnote{\href {https://www.aicrowd.com/challenges/amazon-kdd-cup-2024-multi-task-online-shopping-challenge-for-llms/problems/amazon-kdd-cup-24-multi-lingual-abilities}{Challenge Web}}$. The research introduces a versatile Large Language Model (LLM) based on Qwen2-72B-Instruct, designed to enhance the multi-lingual online shopping experience. Utilizing multi-task learning, the model was fine-tuned to address various tasks derived from Amazon shopping data. To optimize performance, the vLLM library was employed in conjunction with Activation-aware Weight Quantization (AWQ), enabling efficient model inference across four NVIDIA T4 GPUs in the competition environment. This solution demonstrates the potential of LLMs in mastering complex multi-lingual e-commerce tasks, ranging from product navigation to personalized recommendations. The research leverages Qwen2-72B-Instruct as the foundation for fine-tuning, showcasing its effectiveness in tackling multi-lingual e-commerce challenges.
The code and datasets are publicly available in the following GitLab repository:
$\url{https://gitlab.aicrowd.com/fersebasIn/innova-team-amazon-kdd-cup-2024-track-4-4th-position}$
Submission Number: 7
Loading