Data-enabled reduction of the time complexity of iterative solvers

Yuanwei Bin, Xiang I. A. Yang, Samuel J. Grauer, Robert F. Kunz

Abstract: In the field of scientific computing, complex matrices arise from Laplace, Burgers, Kuramoto-Sivashinsky, and Allen-Cahn equations that are not necessarily symmetric positive definite. Computational fluid dynamics, in particular, often deals with pressure Poisson equation. For iterative solvers, time complexity is one of the most critical properties, if not the most critical. Its notation is O(Nα)<math><mi is="true">O</mi><mo stretchy="false" is="true">(</mo><msup is="true"><mrow is="true"><mi is="true">N</mi></mrow><mrow is="true"><mi is="true">α</mi></mrow></msup><mo stretchy="false" is="true">)</mo></math> with N denoting the size of the discretized system and α the scaling exponent. This property indicates how an iterative method's performance scales with the size of the discretized system. Due to the large size of systems in today's scientific computing, methods with lower time complexity are almost always preferred over those with higher time complexity, regardless of the prefactor. This emphasis on time complexity reveals a significant gap in the literature: although the integration of data-enabled methodologies in scientific computing has led to the developments of convergence accelerators and the observation of a speedup of O(10) or so, the reported reductions in cost predominantly concern the prefactor rather than the time complexity. This paper aims to explore reduction in time complexity. The accelerator developed in this paper involves projecting the intermediate solution, which is otherwise only used to assess the residual in the baseline iterative method, onto a low-dimensional Hilbert subspace and directly solving the discretized system there. The solver alternates between the baseline iterative method and the accelerator. Our scaling analysis, which is usually not possible for data-based methods, shows a O(Ni)<math><mi is="true">O</mi><mo stretchy="false" is="true">(</mo><msub is="true"><mrow is="true"><mi is="true">N</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo stretchy="false" is="true">)</mo></math> reduction in the time complexity for Nid<math><msubsup is="true"><mrow is="true"><mi is="true">N</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow><mrow is="true"><mi is="true">d</mi></mrow></msubsup></math>-sized problems in d-dimensional space. Here, Ni<math><msub is="true"><mrow is="true"><mi is="true">N</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub></math> is the number of grids in each dimension, and the system size is N=Nid<math><mi is="true">N</mi><mo linebreak="goodbreak" linebreakstyle="after" is="true">=</mo><msubsup is="true"><mrow is="true"><mi is="true">N</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow><mrow is="true"><mi is="true">d</mi></mrow></msubsup></math>. Consolidated by tests up to 109 degrees of freedom, the present method is shown to offer increasingly more acceleration as the problem size increases, up to 200 times speedup for systems of size 109. Moreover, we demonstrate that the accelerator remains effective for highly nonlinear equations and unstructured grids, yielding similar speedup as for Poisson equation.