Daphne: Multi-Pass Compilation of Probabilistic Programs into Graphical Models and Neural Networks
Abstract: Daphne is a probabilistic programming system that provides an expressive syntax to denote a large, but restricted, class of probabilistic models. Programs written in the Daphne language can be compiled into a general graph data structure of a corresponding probabilistic graphical model with simple link functions that can easily be implemented in a wide range of programming environments. Alternatively Daphne can also further compile such a graphical model into understandable and vectorized PyTorch code that can be used to train neural networks for inference. The Daphne compiler is structured in a layered multi-pass compiler framework that allows independent and easy extension of the syntax by adding additional passes, while leveraging extensive partial evaluation to reduce all syntax extensions to the graphical model at compile time.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: We thank the reviewers for the detailed and encouraging feedback and aimed to address the all concerns with this revision.
In particular we have made the motivation of Daphne more explicit by including two applications of it in the new Section 2, demonstrating why a system translating from high-level syntax to graphical model specifications can be fruitful for probabilistic machine learning research and applications. These papers also compare against a range of existing inference methods and demonstrate that together with their amortized inference artefacts Daphne can provide state-of-the-art inference results on challenging problems. These integrations are not yet fully automated though (WIP). Since submission Daphne has also gained support to compile GPPLs directly to CUDA kernels, and we also aim to provide self-normalized importance sampling and potentially more powerful GPU-based inference methods, but these are not ready yet.
The related work section has also been expanded to include a table of properties that are compared between different PPLs. The section is expanded to highlight the differences of Daphne compared to Anglican and Stan. The limitations have been expanded to concretize in which way expression swell is a fundamental problem and which improvements can still be made. The support for auto-differentiation has also been concretized, and while it is supporting the full target GPPL language, it contains only a limited set of derivative definitions for primitive functions that would need to be extended for more complete support. The difference to the concept of a FOPPL language from which GPPL's and Daphne's definition were derived is also made more precise and hopefully clearer now. Daphne's GitHub code base has documentation of how it is supposed to be run, but the link is not included yet to preserve anonymization.
Further feedback and suggestions for clarifications are welcome!
Assigned Action Editor: ~Andriy_Mnih1
Submission Number: 3262
Loading