Categorical model of neural networks
Keywords: neural networks, systems theory, categorical theory of systems, polycategories, composition
Abstract: A categorical model of neural networks arises naturally when trying to consider neural networks from a systemic point of view. Neural networks are modeled as corresponding systems, which is useful not only from the conceptual point of view of identifying the systemic nature of neural networks. Such modeling turns out to be useful for specific issues, in particular, the substantiation of the well-known formulas of S. Osovsky, used in the method of backpropagation of errors.
Moreover, the categorical theory of systems that we are developing allows us to naturally model not only traditional artificial neural networks of arbitrary topology, but also networks of living neurons, which, in addition to spiking communication, have several dozen other types of cellular communication, and also allows us to model network structures similar to higher categories.
The theory of functional systems by P.K. Anokhin has a categorical nature, which is reflected even in the name. In the postulates of P.K. Anokhin, the construction of the system goes from the whole (the system-forming factor) to the parts, therefore, it is the categorical language that is adequate to the theory of systems. The categorical theory of systems serves as a formalization and development of the general theory of functional systems by P.K. Anokhin.
The neural network is initially considered as a system (in our case, a categorical system), for it, therefore, a whole series of categorical properties are fulfilled, which are considered in this work.
Polycategories were introduced in 1975 by Szabo as a set of polyarrows, the composition of which is defined similarly to the composition of arrows and multiarrows in categories and multicategories. Polyarrows have inputs and outputs, and it is natural to model neurons with them. However, the connections of neurons observed in the brain are significantly richer than the possibilities that can be provided by the composition of Szabo's polycategories. Szabo's theory of polycategories finds applications, but is very complex; working with them "manually", as Szabo does according to R. Garner, encounters problems. For the case of symmetric polycategories, R. Garner constructed their representation in the form of monads in a suitable two-sided Kleisli bicategory. The representation constructed by G. Garner generalizes a similar well-known representation of multicategories in the form of monoids in strut categories. An attempt at a similar generalization for arbitrary polycategories did not lead to the final construction of a representation of polycategories. Szabo introduced polycategories for logic problems, where the inference rules are polyarrows in Gentzen's approach (conjunctions of premises are translated into disjunctions of formulas) and they are connected to each other using one output of the first and one input of the other polyarrow. This restriction in the ways of connecting polyarrows is to some extent removed in PROPs introduced by MacLane, which are used in the categorical approach to networks.
In our approach, we proceed from the needs of modeling the connections of neurons in the brain, for which there are not enough connections in the form of the studied compositions of polyarrows in the Szabo and PROP polycategories, and the theory of systems, when it is necessary not only to assemble a system from future subsystems, but also to decompose it into subsystems. For this, we replace the compositions of polyarrows with a new, more general type of connections called convolutions, introduce and use categorical splices gluings, from which polycategories are built with an explicit assignment of the history of obtaining polyarrows (analogous to a nerve in categories). In the work, new results are obtained, and proofs of previously announced theorems are presented.
Submission Number: 43
Loading