Constraint Reasoning for Structured Prediction and Image Generation

Yexiang Xue, Maxwell J Jacobson, Nan Jiang, Willem-Jan van Hoeve

Abstract:

Building next-generation generalist systems requires machine learning to capture data distribution and constraint reasoning to ensure structure validity.
Nevertheless, effective approaches are lacking in bridging constraint satisfaction and machine learning. We propose COnstraint REasoning embedded structured learning (CORE), a scalable constraint reasoning and machine learning integrated approach for learning over structured domains. We propose to embed the reasoning module as a layer in the sequential neural networks for structured prediction and content generation. We evaluate CORE on several applications: vehicle dispatching service planning, if-then program synthesis, text2SQL generation, and constrained image generation. The proposed CORE module demonstrates superior performance over state-of-the-art approaches in all the applications. The structures generated with CORE satisfy 100% of the constraints, when using exact decision diagrams.