Reproducibility report for "Interpretable Complex-Valued Neural Networks for Privacy Protection"

Arsen Sheverdin, Noud Corten, Alko Knijff, Georg Lange

Keywords: deep neural networks, privacy, confidentiality

Abstract:

Scope of Reproducibility

The original work by Xiang et al. claimed (1) that complex-valued DNNs effectively increase the difficulty of inferring inputs for the adversary attacks compared to the baseline.
In addition, Xiang et al. stated that the (2) proposed privacy-protecting complex-valued DNN effectively preserves the accuracy when compared to the baseline.

Methodology

Since the original paper's code was not published, all of the codebase was written independently from scratch, based solemnly on how it was described in the paper. We mostly used a Nvidia's RTX 2060 Super as the GPU and a AMD Ryzen 3600x as the CPU. The runtime of each model was highly dependant on the architecture used. The runtimes for each model can be found in Table 2

Results

In contrast to the first claim, we have discovered that for most of the architectures, reconstruction errors for the attacks are quite low, which means that in our models the first claim is not supported. We also found that for most of the models, the classification error is somewhat higher than those provided in the paper. However, these indeed relate to the original work and partially support the second claim of the authors.

What was easy

Authors of the original paper utilized famous architectures for some of architectures' parts, such as ResNet and LeNet, that were well explained and defined in the literature. In addition, authors, provided formulas on the modified rotation-invatiant Complex DNN modules (ReLU, max pooling etc.), implementation of which was relatively straightforward. The paper was based on the openly available datasets.

What was difficult

The paper did not provide any information on the architecture of the critic for the WGAN, along with the architecture of the angle discriminator utilized in inversion attack 1. It also does not provide any information about crucial hyperparameters, such as the k value used for k-anonimity.

Communication with original authors

We did not contact the original authors of the publication.

Paper Url: https://openreview.net/forum?id=S1xFl64tDr