Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
GitGraph - from Computational Subgraphs to Smaller Architecture Search Spaces
Kamil Bennani-Smires, Claudiu Musat, Andreea Hossmann, Michael Baeriswyl
Feb 12, 2018 (modified: Feb 12, 2018)ICLR 2018 Workshop Submissionreaders: everyone
Abstract:To simplify neural architecture creation, AutoML is gaining traction - from evolutionary algorithms to reinforcement learning or simple search in a constrained space of neural modules.
A big issues is its computational cost: the size of the search space can easily go above 10^10 candidates for a 10-layer network and the cost of evaluating a single candidate is high - even if it's not fully trained.
In this work, we use the collective wisdom within the neural networks published in online code repositories to create better reusable neural modules. Concretely, we (a) extract and publish GitGraph, a corpus of neural architectures and their descriptions; (b) we create problem-specific neural architecture search spaces, implemented as a textual search mechanism over GitGraph and (c) we propose a method of identifying unique common computational subgraphs.
Keywords:automl, search space, dataset, github
Enter your feedback below and we'll get back to you as soon as possible.