\documentclass[twocolumn]{aastex631}

\usepackage{amsmath}
\usepackage{multirow}
\usepackage{natbib}
\usepackage{graphicx} 
\usepackage{aas_macros}

\begin{document}


Physics-Informed Neural Networks (PINNs) represent a significant advancement in solving partial differential equations (PDEs) by embedding the governing physical laws directly into the neural network architecture and training objective. This approach offers compelling advantages, such as the ability to handle complex geometries and scenarios with limited observational data, providing a mesh-free alternative to traditional numerical techniques. However, despite their successes, PINNs, like many deep learning models, often function as "black boxes," obscuring the precise mechanisms by which they learn and represent the underlying physical phenomena. Understanding how these networks encode complex solution landscapes and incorporate the influence of problem parameters, such as initial and boundary conditions, is paramount for enhancing their reliability, interpretability, and facilitating downstream applications like model compression or transfer learning.

A central element within many neural network architectures, including PINNs, is the latent space. This intermediate representation layer compresses high-dimensional input data into a more abstract, often lower-dimensional, form. In the context of a PINN solving a PDE, the latent space typically holds a learned encoding of the physical state of the system at specific points in space and time $(x, t)$. Investigating the structure of this latent space provides a window into how the network perceives and processes the physics. A fundamental challenge lies in deciphering how the latent representation varies across the physical domain $(x, t)$ and, critically, how this variation changes in response to modifications in the problem's parameters, such as the initial condition. The difficulty is compounded by the potentially high dimensionality of the latent space (10 dimensions in this study) and the unknown, potentially complex non-linear geometric structures formed by the collection of latent vectors corresponding to a given physical solution. For a specific initial condition, the set of latent vectors $\{L(x,t)\}$ sampled over a grid of $(x,t)$ points forms a point cloud in this 10-dimensional space, whose intrinsic structure and relationship to other such point clouds generated by different initial conditions are not \textit{a priori} understood.

This study focuses on dissecting the geometric structure of the 10-dimensional latent space generated by a PINN trained to solve the 2D Burger's equation. The 2D Burger's equation is a canonical non-linear PDE widely used as a simplified model for complex fluid dynamics phenomena like turbulence and shock formation, known for its rich dynamic behavior highly sensitive to initial conditions. We specifically examine how the PINN's latent representation of the solution changes across 25 distinct initial conditions. For each initial condition, we treat the collection of latent vectors $\{L(x,t)\}$ sampled across a discrete grid of $(x,t)$ points as a dataset forming a point cloud in $\mathbb{R}^{10}$. Our primary objective is to analyze the geometric properties of these point clouds, characterizing their effective dimensionality, shape, and how these characteristics compare and contrast across the ensemble of 25 initial conditions. We hypothesize that despite the complexity of the Burger's equation and the high dimensionality of the latent space, the network may learn a structured and perhaps simple encoding where the latent point clouds exhibit low-dimensional geometric properties and are related across initial conditions by simple transformations.

To achieve this, we employ a suite of geometric analysis techniques. Principal Component Analysis (PCA) is utilized extensively to quantify the dominant directions of variation and determine the effective low dimensionality of the latent vector point clouds, both for the global collection of all latent vectors across all initial conditions, and for the point cloud corresponding to each individual initial condition. Furthermore, we employ subspace similarity measures to quantitatively compare the orientations of the principal subspaces learned for different initial conditions. By systematically analyzing the centroids of these point clouds and the relationship between their principal components and the global latent space structure, we aim to build a comprehensive picture of how the PINN encodes the effect of varying initial conditions within its learned representation. This approach allows us to test whether changes in initial conditions correspond to simple, predictable geometric transformations, such as translations or rotations, of a fundamental latent structure.

Our analysis reveals a highly structured organization within the latent space. We find that, while the latent space is 10-dimensional, the entire collection of latent vectors across all initial conditions occupies an effectively 6-dimensional subspace, capturing over 99\% of the total variance. Strikingly, for each individual initial condition, the corresponding set of latent vectors forms a distinct, approximately 3-dimensional affine manifold. This 3D structure is remarkably consistent in its intrinsic dimensionality and variance distribution across all 25 tested initial conditions. Crucially, the primary effect of changing the initial condition is encoded as a translation of this consistent 3D manifold. These manifold centroids trace a nearly one-dimensional path within the 10-dimensional latent space, strongly aligned with the dominant global principal component. Moreover, the orientations of these 3D manifolds are exceptionally similar, exhibiting an average subspace similarity exceeding 0.98, indicating they are nearly parallel with only subtle, low-dimensional variations in their alignment. These findings demonstrate that the PINN learns a highly efficient and structured parameterization where initial conditions select specific, geometrically simple, and highly related low-dimensional structures within the overall latent space, offering valuable insights into the network's internal encoding mechanisms and suggesting potential avenues for model interpretation and compression.


\end{document}
                