ModernTCN Revisited: A Critical Look at the Experimental Setup in General Time Series Analysis

Önder Akacik, Mark Hoogendoorn

Abstract: While numerous time series models claim state-of-the-art performance, their evaluation often relies on flawed experimental setups, leading to questionable conclusions. This study provides a critical re-evaluation of this landscape, using ModernTCN as a case study. We conduct a rigorous and extended benchmark, correcting methodological issues related to data loading, validation, and evaluation methods, and show that performance claims are sensitive to these details. Additionally, we find that ModernTCN overlooks a line of research in global convolutional models, and our comparison reveals that despite claims of an enlarged effective receptive field (ERF), it falls short of these methods. More than a critique, we introduce an architectural innovation: by embedding irregularly sampled data with a continuous kernel convolution and processing it with the ModernTCN backbone, we achieve new state-of-the-art performance on the challenging PhysioNet 2019 dataset. This work not only provides a robust reassessment of ModernTCN but also serves as an audit of the commonly used general time series analysis experimental setup, which includes tasks such as forecasting, imputation, classification, and anomaly detection.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

**Major Changes:**

- **Broadened Scope:**
Included 6 foundational long-term forecasting baselines. In general, now ensuring at least 3 baselines per experiment.

- **Methodological Justifications:**
Added detailed kernel size selection and parameter tuning explanations in Section 3.3 and Appendix A.1.

- **Statistical Significance:**
Added one-sided Wilcoxon signed-rank test for PhysioNet 2019 results.
Reported standard deviations over 5 runs for forecasting and imputation tasks (Appendices C.1 and C.2).

- **Refocused Paper:**
Updated the main content, introduction, abstract, discussion, and title to emphasize auditing general time series analysis setups beyond ModernTCN.

Code: https://github.com/onderakacik/RE-ModernTCN

Assigned Action Editor: ~Jiang_Bian1

Submission Number: 4484