Unsupervised Lifelong Learning with Sustained Representation Fairness

Heng Lian, Xingquan Zhu, My T. Thai, Yi He

Primary Area: transfer learning, meta learning, and lifelong learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Lifelong Learning, Fairness-aware Machine Learning, Unsupervised Learning, Adversarial Games

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Lifelong learning, pivotal in the incremental improvement of decision-making
functions, can be ill-conditioned when dealing with one or several upcoming
tasks that insinuate spurious correlations between target labels and sensitive de-
mographic attributes. This often results in biased decisions, disproportionately
favoring certain demographic groups. Prior studies to de-bias such learners by
fostering fairness-aware, intermediate representations often overlook the inherent
diversity of task distributions, thereby faltering in ensuring fairness in a lifelong
fashion. This challenge intensifies in the context of unlabeled tasks, where dis-
cerning distributional shifts for the adaptation of learned fair representations is
notably intricate. Motivated by this, we propose Sustaining Fair Representations
in Unsupervised Lifelong Learning (FaRULi), a new paradigm inspired by human
instinctive learning behavior. Like human who tends to prioritize simpler tasks
over more challenging ones that significantly outstrip one’s current knowledge
scope, FaRULi reschedules a buffer of tasks based on the proximity of their fair
representations. The learner starts from tasks that share similar fair representa-
tions, accumulating essential de-biasing knowledge from them. Once the learner
revisits a previously postponed task with more disparate demographic distribu-
tions, it is more likely to increment a fair representation from it, as the learner
is now provided a larger rehearsal dataset enriched from the learned tasks with
diverse demographic patterns. FaRULi showcases promising capability in mak-
ing fair yet accurate decisions in a sequence of tasks without supervision labels,
backed by both theoretical results and empirical evaluation on benchmark datasets.
Code is available at: anonymous.4open.science/r/FaRULi/.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2945