This extended abstract reports preliminary results on fine-tuning pre-trained language models for solving reactive synthesis problems end-to-end. In recent work, hierarchical Transformer neural networks have been successfully trained from scratch to synthesize sequential circuits directly out of formal specifications. We improve over existing approaches by fine-tuning CodeT5 models that have been pre-trained on both natural language and programming languages. Our experiments show improved generalization and sample efficiency compared to the previous approach.

Neural Circuit Synthesis with Pre-trained Language Models

Assertion-based verification is a technique to ensure that a circuit design conforms to its specification and help detect errors early in the design process.
It is enabled by powerful industry and open-source model-checking tools that automatically prove or disprove an assertion for a given circuit design.
Formalizing a circuits requirement, however, involves a significant manual effort by verification engineers to translate requirements in natural language into a formal assertion language.
In this extended abstract, we introduce a framework that utilizes Large Language Models (LLMs) pre-trained on natural language and code to improve verification productivity by automating the formalization process.
In particular, we report on the current progress of developing \texttt{nl2sva}, a framework for circuit-aware translations of natural language to the most frequently used assertion language, SystemVerilog Assertions (SVA).

Towards Improving Verification Productivity with Circuit-Aware Translation of Natural Language to SystemVerilog Assertions

Solving parity games is a major building block for numerous
applications in reactive program verification and synthesis.
While they can be solved efficiently in practice, no known
approach has a polynomial worst-case runtime complexity.
We present a incomplete polynomial-time approach to de-
termining the winning regions of parity games via graph
neural networks.
Our evaluation on 900 randomly generated parity games
shows that this approach is effective and efficient in prac-
tice. It correctly determines the winning regions of ∼60% of
the games in our data set and only incurs minor errors in
the remaining ones. We believe that this approach can be
extended to efficiently solve parity games as well.

Predicting Winning Regions in Parity Games via Graph Neural Networks

This paper presents a tool that can generate system model and temporal logic formula pairs such that the formula is satisfied or unsatisfied as desired. Large amounts of independent and identically distributed data is needed for the development of machine learning approaches in formal methods. However, existing data sets of system models and specifications were hand-designed to challenge existing algorithms for formal methods, and so these data sets represent only a small subset of problems. Learning-based approaches to formal methods require training data drawn from a broader distribution. In this work we introduce a tool called ``FoMo'' (for ``Formula and Model'') that generates system models from graph distributions and can sample a specification language to generate properties. FoMo includes functions to generate pairs of formulas and satisfying or unsatisfying system models, and to generate traces from systems. The tool has features for working with linear temporal logic, and with weighted automata. We demonstrate the use of this tool by training a neural network to jointly embed formulas and models to allow for model checking as a classification task. The capabilities offered by FoMo allow researchers to generate system models and properties according to many research needs, and helps provide machine learning systems with generated, multi-modal data sets.

FoMo - Formula and Model Generation for Learning-Based Formal Methods

Boolean satisfiability problem (SAT) is fundamental to many applications. Existing works have used graph neural networks (GNNs) for (approximate) SAT solving. Typical GNN-based end-to-end SAT solvers predict SAT solutions concurrently.  We show that for a group of symmetric SAT problems, the concurrent prediction is guaranteed to produce a wrong answer because it neglects the dependency among Boolean variables in SAT problems. We propose AsymSAT, a GNN-based architecture which integrates recurrent neural networks to generate dependent predictions for variable assignments. The experiment results show that dependent variable prediction extends the solving capability of the GNN-based method as it improves the number of solved SAT instances on large test sets.

CAV 2023 Workshop DAV Submissions

Neural Circuit Synthesis with Pre-trained Language Models

Towards Improving Verification Productivity with Circuit-Aware Translation of Natural Language to SystemVerilog Assertions

Predicting Winning Regions in Parity Games via Graph Neural Networks

FoMo - Formula and Model Generation for Learning-Based Formal Methods

The Elephant in the Room: Variable Dependency in GNN-based SAT Solving