Extrapolation of Periodic Functions Using Binary Encoding of Continuous Numerical Values

TMLR Paper7144 Authors

Abstract: We report the discovery that binary encoding allows neural networks to extrapolate periodic functions beyond their training bounds. We introduce Normalized Base-2 Encoding (NB2E) as a method for encoding continuous numerical values and demonstrate that, using this input encoding, vanilla multi-layer perceptrons (MLP) successfully extrapolate diverse periodic signals without prior knowledge of their functional form. Internal activation analysis reveals that NB2E induces bit-phase representations, enabling MLPs to learn and extrapolate signal structure independently of position.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=P7woOwzyif

Changes Since Last Submission: Removed acknowledgements section for double blind anonymization.

Assigned Action Editor: ~Yunhe_Wang1

Submission Number: 7144