ResearchArena-CayleyBench: RL/LLM Benchmark challenges which can advance mathematical research

Alexander Chervov, Dmytro Fedoriaka, Mark Obozov, Elena V. Konstantinova, Anastasia Sheveleva, Alexander Soibelman, Igor Kiselev, Fedor Levkovich-Maslyuk, Dmitry Volovich, Nick Vilkin-Krom, Alim Bidzhiev, Artem Krasnyi, Mikhail Evseev, Artem Isakov, Michael Litvinov, Elizaveta Geraseva, Liliya Grunwald, Sergei Lytkin, Andrei Smolensky, Sergey Galkin et al. (29 additional authors not shown)

Abstract:

This is the third paper of the CayleyPy project applying artificial intelligence methods to problems in group theory.
We announce the first public release of CayleyPy, an open-source Python library for computations with Cayley and Schreier (coset) graphs. Compared with state-of-the-art systems based on classical methods, such as GAP and Sage, CayleyPy handles significantly larger graphs and performs many orders of magnitude times faster.

Using CayleyPy we obtained about 200 new mathematical conjectures on Cayley and Schreier graphs which can be turned into an efficient benchmarks for both RL and LLM models. For many Cayley graphs of symmetric groups $S_n$ we observe quasi-polynomial diameter formulas: a small set of quadratic or linear polynomials indexed by $n \mod s$ and conjecture that it is general phenomenon. We conjecture improved Babai-type bounds of the diameters by $\frac12 n^2 + 4n$ for undirected case, by $\frac34 n^2 + O(n)$ for directed cases, and by $\frac14 n^2 + O(n)$ for certain Schreier graphs, comparing to prior conjectural bounds of $O(n^2)$. For nilpotent groups we conjecture an improvement of J.S. Ellenberg's results on the diameter of the upper-triangular matrices over $\Z/p$, presenting a phenomenon of linear dependence of the diameter with respect to $p$. Moreover, the growth for nilpotent groups is conjectured to follow Gaussian distributions, that is, to exhibit a central limit phenomenon similar to results of P. Diaconis for $S_n$.