Towards Fair and Robust Face Parsing for Generative AI: A Multi-Objective Approach

Sophia Abraham, Jonathan Hauenstein, Walter Scheirer

Keywords: Fairness in AI, Robust Face Parsing, Multi-Objective Optimization, Generative Adversarial Networks, Diffusion Models, Demographic Bias

TL;DR: We propose a multi-objective learning framework that improves fairness, robustness, and accuracy in face parsing models, leading to more equitable and photorealistic generative AI outputs.

Abstract:

Face parsing is a fundamental task in computer
vision, enabling applications such as identity verification, facial
editing, and controllable image synthesis. However, existing face
parsing models often lack fairness and robustness, leading to
biased segmentation across demographic groups and errors
under occlusions, noise, and domain shifts. These limitations
affect downstream face synthesis, where segmentation biases
can degrade generative model outputs. We propose a multi-
objective learning framework that optimizes accuracy, fairness,
and robustness in face parsing. Our approach introduces a
homotopy-based loss function that dynamically adjusts the
importance of these objectives during training. To evaluate its
impact, we compare multi-objective and single-objective U-Net
models in a GAN-based face synthesis pipeline (Pix2PixHD).
Our results show that fairness-aware and robust segmenta-
tion improves photorealism and consistency in face genera-
tion. Additionally, we conduct preliminary experiments using
ControlNet, a structured conditioning model for diffusion-
based synthesis, to explore how segmentation quality influences
guided image generation. Our findings demonstrate that multi-
objective face parsing improves demographic consistency and
robustness, leading to higher-quality GAN-based synthesis.

Submission Number: 12