GitGraph - from Computational Subgraphs to Smaller Architecture Search Spaces

Kamil Bennani-Smires, Claudiu Musat, Andreea Hossmann, Michael Baeriswyl

Abstract:

To simplify neural architecture creation, AutoML is gaining traction - from evolutionary algorithms to reinforcement learning or simple search in a constrained space of neural modules.
A big issues is its computational cost: the size of the search space can easily go above 10^10 candidates for a 10-layer network and the cost of evaluating a single candidate is high - even if it's not fully trained.
In this work, we use the collective wisdom within the neural networks published in online code repositories to create better reusable neural modules. Concretely, we (a) extract and publish GitGraph, a corpus of neural architectures and their descriptions; (b) we create problem-specific neural architecture search spaces, implemented as a textual search mechanism over GitGraph and (c) we propose a method of identifying unique common computational subgraphs.

Keywords: automl, search space, dataset, github