A Scalable Approach to Probabilistic Neuro-Symbolic Robustness Verification

Vasileios Manginas, Nikolaos Manginas, Edward Stevinson, Sherwin Varghese, Nikos Katzouris, Georgios Paliouras, Alessio Lomuscio

Keywords: Neuro-symbolic methods, safety and robustness

Abstract: Neuro-Symbolic Artificial Intelligence (NeSy AI) has emerged as a promising direction for integrating neural learning with symbolic reasoning. Typically, in the probabilistic variant of such systems, a neural network first extracts a set of symbols from sub-symbolic input, which are then used by a symbolic component to reason in a probabilistic manner towards answering a query. In this work, we address the problem of formally verifying the robustness of such NeSy probabilistic reasoning systems, therefore paving the way for their safe deployment in critical domains. We analyze the complexity of solving this problem exactly, and show that a decision version of the core computation is $\mathrm{NP}^{\mathrm{PP}}$-complete. In the face of this result, we propose the first approach for approximate, relaxation-based verification of probabilistic NeSy systems. We demonstrate experimentally on a standard NeSy benchmark that the proposed method scales exponentially better than solver-based solutions and apply our technique to a real-world autonomous driving domain, where we verify a safety property under large input dimensionalities.

Track: Main Track

Paper Type: Long Paper

Resubmission: Yes

Software: https://github.com/EVENFLOW-project-EU/nesy-veri

Changes List:

This paper is a NeSy Phase 1 resubmission. We would like to thank the reviewers for the useful comments, which helped us improve the paper. Below, we explain how we have addressed the reviewers' comments.

[REVIEWER 1:]
[COMMENT:] The most problematic issue is the claim of NP^#p hardness. The argument is not an argument for hardness [since] there is no reduction to the problem that is claimed to be NP^#p hard.
[RESPONSE:] In the revised version of the paper we prove (Section 3.2 & Appendix B) that a decision version of our problem, which lower-bounds the actual functional version, is NP^PP-complete. We prove hardness via a reduction from E-MAJSAT, the prototypical NP^PP-complete problem.

[COMMENT:] The paper could delve more deeply into the literature of knowledge compilation, specifically with regards to the size of the circuits obtained, and the corresponding effects on the proposed approach.
[RESPONSE:] To obtain proof-of-concept results for our work we utilised the Sentential Decision Diagram (as mentioned in Appendix A), a standard boolean circuit type within the NeSy literature. An in-depth analysis of the effect of different circuit types and sizes on the efficacy of our method is a good suggestion for future work.

[COMMENT:] It is not clearly stated for what logical formalisms the proposed methods applies.
[RESPONSE:] Our method only requires that the logic can be compiled into an arithmetic circuit. For example, we support the full expressive power of logic programming under probabilistic semantics by compiling logic programs into arithmetic circuits via knowledge compilation. We have updated the Related Work section to reflect this.

[COMMENT:] The conceptual contribution and novelty should be discussed in more detail.
[RESPONSE:] We have updated the Introduction and the Related Work sections to better reflect the novelty and conceptual contribution of our work.

-----------------------

[REVIEWER 2:]
[COMMENT:] The phrase "verification of NeSy systems" and the corresponding paper title is an inherited misnomer since the work (only) verifies robustness.
[RESPONSE:] We have modified the title of the paper as per the reviewer’s suggestion.

[COMMENT:] Why is the bound in the final sentence of section 3.1 equal to 0.5? Is this an arbitrary lower bound on the probability of the constraints holding?
[RESPONSE:] We now mention in the text that the choice of the probabilistic threshold for robustness is tunable. We use a default value of 0.5, signifying that for all ε-perturbed inputs it’s more likely than not that the constraints will hold.

[COMMENT:] It is nice to precisely know the behaviour of the symbolic component. You lose this by extending an approximate NN verifier to the whole NeSy system.
[RESPONSE:] The use of approximate techniques in the context of our work does not retract the ability to precisely know the behaviour of the symbolic component in terms of “normal” inference (i.e. without perturbations).

-----------------------

[REVIEWER 3:]
[COMMENT:] The comparison with prior neuro-symbolic verification methods is somewhat narrow, focusing mainly on solver-based tools.
[RESPONSE:] The choice of methods against which we compare the proposed approach was the result of an extensive study of existing approaches. We believe that the ones we have chosen best depict the state-of-the-art, which does not currently contain any non-solver-based methods.

[COMMENT:] More insight into the trade-offs between relaxation accuracy and robustness certification could further strengthen the evaluation.
[RESPONSE:] We are not certain of the meaning of this comment. In general, the soundness of relaxation-based methods entails that they can never output a false positive robustness certification. Thus, higher relaxation accuracy can only lead to higher robustness certification (with the upper bound being the robustness certification of an exact method). As such, there can only be a positive correlation, and no trade-off, between the two quantities mentioned in the comment.

Submission Number: 82