Correctness Verification of Neural Network

Yichen Yang, Martin Rinard

Keywords: Neural network verification, safety, reliability

TL;DR: We present the first verification that a neural network for perception tasks produces a correct output within a specified tolerance for every input of interest.

Abstract:

We present the first verification that a neural network for perception tasks produces
a correct output within a specified tolerance for every input of interest. We define
correctness relative to a specification which identifies 1) a state space consisting of
all relevant states of the world and 2) an observation process that produces neural
network inputs from the states of the world. Tiling the state and input spaces with
a finite number of tiles, obtaining ground truth bounds from the state tiles and
network output bounds from the input tiles, then comparing the ground truth and
network output bounds delivers an upper bound on the network output error for
any input of interest. Results from two case studies highlight the ability of our
technique to deliver tight error bounds for all inputs of interest and show how the
error bounds vary over the state and input spaces.

Code: https://anonymous.4open.science/r/5f526d25-cdbf-46db-b737-b235676481b7/