Relation-First Modeling Paradigm for Causal Representation Learning toward the Development of AGI
Abstract: The traditional i.i.d.-based learning paradigm faces inherent challenges in addressing causal relationships, which has become increasingly evident with the rise of applications in causal representation learning.
Our understanding of causality naturally requires a perspective as the creator rather than observer, as the "what...if" questions only hold within the possible world we conceive. The traditional perspective limits capturing dynamic causal outcomes and leads to compensatory efforts such as the reliance on hidden confounders.
This paper lays the groundwork for the new perspective, which enables the relation-first modeling paradigm for causality. Also, it introduces the Relation-Indexed Representation Learning (RIRL) as a practical implementation, supported by experiments that validate its efficacy.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=EzOJkQk4Ar
Changes Since Last Submission: #------------ Revision on 02/28/2024 -------------#
Thoroughly improved expressions.
#------------ Revision on 02/23~27/2024 -------------#
Revised interpretations around Figure 4 and Figure 6.
#------------ Revision on 02/15/2024 -------------#
The content of this paper, including the title, has been thoroughly revised.
I would like to extend my heartfelt thanks to Reviewer wRNQ for providing critical guidance and references that point me toward the relevant philosophical literature.
Given my advisor's antipathy towards philosophical discussions, I had grown accustomed to steering clear of philosophy, which kept me at a distance from truly clarifying my theory and led to chaotic presentations. Your insights were invaluable to me.
The current version has been significantly improved and streamlined.
Looking forward to your feedback, and thanks again!
.
.
######## Brief Description for Review (Added on 12/9) #########
After receiving the first review, I realized that a brief introduction might help reviewers more easily grasp the central theme.
Additionally, as a non-native English speaker and a foreigner, I earnestly seek your guidance on _whether_ and _how to_ include the following content in the paper without offending potential readers from diverse backgrounds.
The essence of causality lies in ___temporally nonlinear__ (i.e., dynamical) effects_, and realizing AGI involves fulfilling them as _causal representations_. This necessitates a reevaluation and potential overhaul of existing causal inference theories and even the foundational learning paradigm, which is the objective of this paper. The current "Observation-Oriented" paradigm encompasses all models based on i.i.d. observations, including AI-driven ones like RNNs, LLMs, image recognition, etc.
About why causal inference is off, consider these:
* It's known that causality vs. correlation cannot be distinguished solely by models. If such distinction is insignificant for the model, why is it emphasized when interpreting the model? If it is critical for the model, why such significance cannot be reflected by models? Isn't there something wrong?
* Many scholars have been perplexed by inherently counter-intuitive elements, such as using a hidden confounder to illogically present "our model is biased due to something unknown, which we do not intend to know." But what made us rely on these elusive concepts for decades?
* Is it supposed that causal inference and AI models (like LLMs) are independent? If so, what shall we expect for AI with causality, i.e., the AGI?
The existing "causal representation" is not directly borrowed in this paper, which broadly encompasses "relational (or simply correlational) representation".
#################################################\
.\
#--------- submitted on 11/16 -----------
This rewritten version features a completely restructured organization to enhance reader-friendliness.
Additionally, new overview sections have been included to ensure a more streamlined and coherent presentation.
Special thanks are due to Reviewer Jqrm, whose valuable suggestions significantly contributed to these improvements.
Please kindly ensure that Reviewer Jqrm is also invited to review this submission.
Thank you very much!
#--------- update on 11/22 -----------
This version has revised Section 2 and Section 3 for more rigorous definitions and better organization.
#--------- update on 11/27 -----------
Revisions in Sections 2 and 8 to clarify the counterfactual essence of $\mathbb{R}^T$.
#--------- update on 12/03 -----------
Revision in section 4.1 for clearly defining _Interaction_, _Dependence_, and _Confounding_ between dynamics. (No further revision before receiving review comments)
#--------- final update on 12/06 -----------
(This is truly the final update... sorry for reneging) Improved writing and minor corrections.
Assigned Action Editor: ~ERIC_EATON1
Submission Number: 1837
Loading