A Critical Study of What Pre-trained Code Models (do not) Learn

Abhinav Anand, Krishna Narasimhan, Mira Mezini

Primary Area: visualization or interpretation of learned representations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Explainability, interpretation of learned representations, neural code intelligence, neural networks

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We study various models pre-trained on code corpora and show that they do not encode sufficient information to understand subtle differences in code syntax.

Abstract: Parallel to the recent success of self-attention-based language models across a range of coding assistance tasks, several studies have underscored that pre-trained code models (PCMs) utilize self-attention and hidden representations to encode relations among input tokens. Our research extends upon these insights by understanding the properties of code that PCMs may not fully encode and by broadening the scope to encompass data flow relations. Our study reveals that while PCMs do encode syntactic and data flow relations in self-attention, they only encode relations within specific subsets of input tokens. Specifically, by categorizing input tokens into syntactic tokens and identifiers, we find that models encode relations among syntactic tokens and among identifiers but fail to encode relations between syntactic tokens and identifiers. We show that this limitation results in hidden representations not encoding enough information about input tokens to discriminate between different identifier types and syntax structures. Importantly, we observe that this learning gap persists across different model architectures, datasets, and pre-training objectives. Our findings shed light on why PCMs fail to generalize beyond dataset they are trained on and in real world applications.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7451