Subjective Learning for Open-Ended Data

Tianren Zhang, Yizhou Jiang, Xin Su, Shangqi Guo, Feng Chen

Keywords: Open-ended data, machine learning, supervised learning, data conflict

Abstract: Conventional supervised learning typically assumes that the learning task can be solved by learning a single function since the data is sampled from a fixed distribution. However, this assumption is invalid in open-ended environments where no task-level data partitioning is available. In this paper, we present a novel supervised learning framework of learning from open-ended data, which is modeled as data implicitly sampled from multiple domains with the data in each domain obeying a domain-specific target function. Since different domains may possess distinct target functions, open-ended data inherently requires multiple functions to capture all its input-output relations, rendering training a single global model problematic. To address this issue, we devise an Open-ended Supervised Learning (OSL) framework, of which the key component is a subjective function that allocates the data among multiple candidate models to resolve the "conflict'' between the data from different domains, exhibiting a natural hierarchy. We theoretically analyze the learnability and the generalization error of OSL, and empirically validate its efficacy in both open-ended regression and classification tasks.

One-sentence Summary: We formalize the problem of learning from open-ended data that implicitly comes from multiple domains and inherently requires multiple functions to fully capture its input-output relations.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2108.12113/code)