Integrating Deep Neural Networks with Dependent Type Semantics

Daisuke Bekki, Ribeka Tanaka, Yuta Takahashi

Abstract: Recent studies in computational semantics have explored how the advantages of both type-logical semantics and deep neural networks can be combined to develop a theory or system of natural language understanding that can perform complex inferences while being learnable. In this study, we propose a theory of language understanding that fuses neural language models and dependent type semantics (DTS), a proof-theoretic semantics of natural language based on dependent type theory, by replacing names and predicates in DTS with tensor-based distributional representations and neural classifiers over them. Under this unified view, interesting correspondences are found between proof-theoretic and machine learning notions. For instance, the value of the loss function provides proof for an atomic predicate, and the neural parameters are regarded as proof-theoretic assumptions about the real world.

External IDs:doi:10.1007/978-3-031-21780-7_11