Emergent Possibilities and Challenges in Deep Learning for Code

Zijian Wang, Ying Sheng, Giovanni Zappella, Qian Liu, Devjeet Raj Roy, Gabriel Orlanski, Zhiruo Wang, Wen-Ding Li

Keywords: Deep Learning for Code, Agentic Methods for Programming Tasks, Post-training and Alignment for Code, Developer Productivity and HCI for Code, Open Science and Responsible AI for Code, Benchmarking and Evaluation for Code

Abstract:

The application of deep learning in code, spanning a diverse set of challenging tasks such as code completion, repair, synthesis, and explanation, constitutes a significant area of research within machine learning. The last two years have witnessed remarkable progress in this domain with the development of recent techniques (Rozière et al., 2023; Li et al., 2023; Guo et al., 2024; Wei et al., 2024; Yang et al., 2024a) that enables us to solve realistic and complex software development tasks. Moreover, code generation is increasingly being used as a critical step in solving complex tasks beyond code, such as reasoning (Yang et al., 2024b; Gao et al., 2023), sequential decision-making (Zhang et al., 2024; Wang et al., 2024a), robotic control (Liang et al., 2023) and machine learning for science and algorithmic discovery (Romera-Paredes et al., 2023; Mankowitz et al., 2023).

These advances underscore the increasingly pivotal role deep learning for code has come to play within the broader machine learning discipline, and have given rise to novel and increasingly complex challenges. The third Deep Learning for Code workshop (DL4C, https://dl4c.github.io) aims to provide a forum for researchers, practitioners, and industry profes- sionals to share insights, collaborate, and advance the state-of-the-art in this raplidly evolving field. Building on the success of previous iterations (ICLR’22, ICLR’23), which showcased significant works through poster sessions, invited talks, and panel discussions from leading experts, the third DL4C workshop will focus on emergent possibilities and challenges in deep learning for code. The key focus areas of this year include agentic methods for programming tasks, post-training and alignment for code, human-computer/AI interaction and developer productivity for code, open science and responsible AI for code, and benchmarking for emergent tasks. By addressing these cutting-edge topics, we aim to solidify our position as a prominent platform for advancing research in the increasingly critical area of deep learning for code.

Submission Number: 88