PyAFAR: Python-based Automated Facial Action Recognition library for use in Infants and Adults

Saurabh Hinduja, Itir Önal Ertugrul, Maneesh Bilalpur, Daniel S. Messinger, Jeffrey F. Cohn

Abstract: PyAFAR1 is a Python-based, open-source facial action unit detection library for use with adults and infants. Convolutional neural networks were trained on BP4D+ for adults and MIAMI and CLOCK databases for infants. In adults, AU occurrence and intensity detection are enabled for 12 action units (AU). The AU chosen were selected on the criterion that they have base rates greater than 5% in BP4D+. Action unit intensity estimation is enabled for 5 of these AU. In infants, AU occurrence is enabled for 9 action units that are involved in expression of positive and negative affect. For both adults and infants, facial landmark and head pose tracking are enabled as well. For adults, multiple persons within a video may be tracked. The library is developed for ease of use. The models are available for fine-tuning and further training. PyAFAR may be easily incorporated into user Python code.1Code Available on: https://github.com/AffectAnalysisGroup/PyAFAR