PIs: Yugi Lee, Andy Li

Deep Learning has improved the state-of-the-art results in an ever-growing number of domains. This success heavily relies on the building of deep learning models. Often, building a model is an ad-hoc, iterative process that results in producing tens to hundreds of models before arriving at a satisfactory result. While there has been a surge in the number of tools and frameworks that aim at facilitating deep learning, the issues of model management have been largely ignored. In particular, deep learning practitioners have to manually track their experiments using text files, spreadsheets or folder hierarchies, which is expensive, time-consuming, and error-prone. We propose the ModelKB system automating end-to-end model management in deep learning. We will develop a ModelKB prototype that can automatically (1) extract and store the model’s metadata-including its architecture, weights, and configuration; (2) visualize, query, and compare experiments; and (3) reproduce experiments.