Edge-LLMs: Edge-Device Large Language Model Competition

Shiwei Liu, Kai Han, Adriana Fernandez-Lopez, AJAY KUMAR JAISWAL, Zahra Atashgahi, Boqian Wu, Edoardo Ponti, Callie Hao, Rebekka Burkholz, Olga Saukh, Lu Yin, Andreas Zinonos, Tianjin Huang, Jared Tanner, Yunhe Wang

Keywords: Deep Learning, Large Language Models, Edge LLM, Model Compression, Edge Computing

TL;DR: We propose this challenge aiming to push the boundaries of what these powerful LLMs can achieve on edge devices in terms of performance, efficiency, and versatility.

Abstract: The Edge-Device Large Language Model Competition seeks to explore the capabilities and potential of large language models (LLMs) deployed directly on edge devices. The incredible capacity of LLMs makes it extremely tantalizing to be applied to practical edge devices to enable wide applications of LLMs in various disciplines. However, the massive size of LLMs poses significant challenges for edge devices where the computing resources and memory are strictly limited. For instance, deploying a small-scale 10B LLM could require up to 20GB of main memory (DRAM) even after adopting INT8 quantization, which unfortunately has exceeded the memory of most commodity smartphones. Besides, the high energy consumption of LLMs will drain smartphones' battery quickly. To facilitate applications of LLMs in a wide range of practical scenarios, we propose this timely competition to encourage practitioners in both academia and industry to come up with effective solutions for this pressing need. By challenging participants to develop efficient and optimized models that can run on resource-constrained edge devices, the competition aims to address critical economic and environmental issues related to LLMs, foster interdisciplinary research collaborations, and enhance the privacy and security of AI systems.

Competition Timeline:

| Date | Event |
| ----------- | ----------- |
| 25th June, 2024 | Announcement and registration start, ``starting kit'' will be released |
| 25th July, 2024 | Registration deadline and submission open |
| 25th August, 2024 | Preliminary review deadline |
| 25th October, 2024 | Submission deadline |
| 20th November, 2024 | Winners notification |
| 11th December, 2024 | In-person workshop |

Website: https://edge-llms-challenge.github.io/edge-llm-challenge.github.io/

Primary Contact Email: [e-mail]

Participant Contact Email: [e-mail]

Workshop Format: In-person (Vancouver)

Preferred Timezone: British Summer Time

Submission Number: 14