Dynamic Structure Estimation from Bandit Feedback using Nonvanishing Exponential Sums

Motoya Ohnishi, Isao Ishikawa, Yuko Kuroki, Masahiro Ikeda

Abstract: This work tackles the dynamic structure estimation problems for periodically behaved discrete dynamical system in the Euclidean space. We assume the observations become sequentially available in a form of bandit feedback contaminated by a sub-Gaussian noise. Under such fairly general assumptions on the noise distribution, we carefully identify a set of recoverable information of periodic structures. Our main results are the (computation and sample) efficient algorithms that exploit asymptotic behaviors of exponential sums to effectively average out the noise effect while preventing the information to be estimated from vanishing. In particular, the novel use of the Weyl sum, a variant of exponential sums, allows us to extract spectrum information for linear systems. We provide sample complexity bounds for our algorithms, and we experimentally validate our theoretical claims on simulations of toy examples, including Cellular Automata.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

Following the reviewers' comments, we revised the manuscript:
1. We reduced the redundancy in introduction
2. We added the motivation section
3. We improved the related work section to better place our work in the literature
4. We added a realistic simulation experiment
5. We added discussions on optimality and parameter selection
6. We added some brief notes on the properties of exponential sums and the research in number theory community
7. We fixed some typos

Video: https://sites.google.com/view/dsefbf/

Code: https://sites.google.com/view/dsefbf/

Assigned Action Editor: ~Murat_A_Erdogdu1

Submission Number: 1979