\documentclass[twocolumn]{aastex631}

\usepackage{amsmath}
\usepackage{multirow}
\usepackage{natbib}
\usepackage{graphicx} 
\usepackage{aas_macros}

\begin{document}

This study investigated the geometric structure of the 10-dimensional latent space generated by a Physics-Informed Neural Network (PINN) trained to solve the 2D Burger's equation across a set of 25 distinct initial conditions. Our goal was to understand how the PINN encodes the physical state of the system and how variations in the initial condition are reflected in the network's internal representation. We hypothesized that the latent space might exhibit a structured, potentially low-dimensional, organization related to the problem parameters.

To address this, we employed Principal Component Analysis (PCA) and subspace similarity measures to analyze the collections of latent vectors. We performed PCA on the entire dataset of latent vectors (global PCA), on the latent vectors for each individual initial condition (per-IC PCA), and on the centroids of the per-IC latent point clouds. Subspace similarity was used to compare the orientations of the principal subspaces identified by the per-IC PCA. The dataset comprised 10-dimensional latent vectors extracted from a pre-trained PINN solution for 25 initial conditions, sampled over a spatial and temporal grid.

Our analysis yielded several key findings regarding the geometric organization of the latent space. Globally, the latent vectors across all initial conditions occupy an effectively 6-dimensional subspace, capturing over 99\% of the total variance, indicating an overall efficient representation. More specifically, for each individual initial condition, the set of latent vectors forms a distinct, approximately 3-dimensional affine manifold embedded within the 10-dimensional space. This intrinsic dimensionality and the distribution of variance along the principal components were remarkably consistent across all 25 initial conditions. Crucially, the primary effect of changing the initial condition is encoded as a translation of this consistent 3D manifold within the latent space. The centroids of these manifolds trace a nearly one-dimensional path, strongly aligned with the dominant global principal component, as the initial condition changes. Furthermore, the 3D manifolds for different initial conditions are remarkably parallel to each other, exhibiting an average subspace similarity exceeding 0.98, with only subtle, low-dimensional variations in their orientation.

From these results, we learned that the PINN develops a highly structured and geometrically simple representation of the Burger's equation solutions. Instead of learning entirely distinct high-dimensional representations for each initial condition, the network appears to learn a fundamental, low-dimensional (3D) structure representing the spatiotemporal evolution of the system for a fixed initial state. The specific initial condition then acts primarily as a parameter that translates this base structure along a specific direction in the latent space. This suggests a form of disentangled representation, where the network separates the influence of the initial condition (encoded as a translation) from the intrinsic dynamics (encoded within the 3D manifold structure). This geometric organization is highly efficient and offers valuable insights into the network's internal encoding mechanisms, suggesting that the PINN captures the essential physics in a structured and potentially interpretable manner within this latent space.

While our findings reveal a compelling geometric structure, it is important to acknowledge the limitations of relying primarily on linear techniques like PCA, which might miss finer non-linear structures. Future work could explore non-linear manifold learning techniques to further probe the geometry. Expanding the study to include variations in physical parameters, such as viscosity, and analyzing a larger, more diverse set of initial conditions would be crucial to assess the generalizability of these findings and potentially uncover more complex organizational principles. Correlating specific properties of the initial conditions with the latent space features (centroid position, manifold orientation) would provide deeper physical meaning. Finally, investigating whether similar structured latent spaces are learned by PINNs for other types of PDEs and with different network architectures is essential to determine the broader applicability of these observed geometric principles.

\end{document}
                