On Trojan Signatures in Large Language Models of Code

Aftab Hussain, Md Rafiqul Islam Rabin, Amin Alipour

Keywords: trojan signatures, code models

Abstract:

Trojan signatures, as described by Fields et al. (2021), are noticeable differences
in the distribution of the trojaned class parameters (weights) and the non-trojaned
class parameters of the trojaned model, that can be used to detect the trojaned model.
Fields et al. (2021) found trojan signatures in computer vision classification tasks
with image models, such as, Resnet, WideResnet, Densenet, and VGG. In this
paper, we investigate such signatures in the classifier layer parameters of large
language models of source code. Our results suggest that trojan signatures could
not generalize to LLMs of code. We found that trojaned code models are stubborn,
even when the models were poisoned under more explicit settings (finetuned with
pre-trained weights frozen). We analyzed nine trojaned models for two binary
classification tasks: clone and defect detection. To the best of our knowledge, this
is the first work to examine weight-based trojan signature revelation techniques
for large-language models of code and furthermore to demonstrate that detecting
trojans only from the weights in such models is a hard problem.

Submission Number: 49