Provable Repair of Vision Transformers: Last Layer is All You Need

Stephanie Nawas, Zhe Tao, Aditya Thakur

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: neural network repair, vision transformers, formal guarantees

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: An architecture-preserving provable repair approach for Vision Transformers that repairs thousands of points by editing the last layer and still achieving low drawdown.

Abstract:

Vision Transformers have emerged as state-of-the-art image recognition tools, but may still exhibit incorrect behavior. Incorrect image recognition can have disastrous consequences in safety-critical real-world applications such as self-driving automobiles. In this paper, we present Provable Repair of Vision Transformers (PRoViT), a provable repair approach that guarantees the correct classification of
images in a repair set for a given Vision Transformer without modifying its ar-
chitecture. PRoViT avoids negatively affecting correctly classified images (draw-
down) by minimizing the changes made to the Vision Transformer’s parameters
and original output. We observe that for Vision Transformers, unlike for other
architectures such as ResNet or VGG, editing just the parameters in the last layer
achieves correctness guarantees and very low drawdown. We introduce a novel
method for editing these last-layer parameters that enables PRoViT to efficiently
repair state-of-the-art Vision Transformers for thousands of images, far exceeding
the capabilities of prior provable repair approaches.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8836