Convergence Analysis of Tsetlin Machines under Noise-Free and Noisy Training Conditions: from $2$ Bits to $k$ Bits

Xuan Zhang, Lei Jiao, Ole-Christoffer Granmo

Keywords: Tsetlin Automata, Propositional Logic, Tsetlin Machine, Convergence Analysis

Abstract: The Tsetlin Machine (TM) is an innovative machine learning algorithm grounded in propositional logic, achieving state-of-the-art performance across a variety of pattern recognition tasks. Prior theoretical work has established convergence results for the 1-bit operator under both noisy and noise-free conditions, and for the 2-bit XOR operator under noise-free conditions. This paper first extends the analysis to the 2-bit AND and OR operators. We show that the TM converges almost surely to the correct 2-bit AND and OR operators under noise-free training, and we identify a distinctive property of the 2-bit OR case, where a single clause can jointly represent two sub-patterns, in contrast to the XOR operator. We further investigate noisy training scenarios, demonstrating that mislabelled samples prevent exact convergence but still permit efficient learning, whereas irrelevant variables do not impede almost-sure convergence. Building on the 2-bit analysis, we then generalize the results to the $k$-bit setting ($k>2$), providing a unified treatment applicable to general scenarios. Together, these findings provide a robust and comprehensive theoretical foundation for analysing TM convergence.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 14096