Visual Ensemble Analysis to Study the Influence of Hyper-parameters on Training Deep Neural Networks

Hamid, Sagad; Derstroff, Adrian; Klemm, Sören; Ngo, Quynh Quang; Jiang, Xiaoyi; Linsen, Lars

Visual Ensemble Analysis to Study the Influence of Hyper-parameters on Training Deep Neural Networks

Files

019-023.pdf (627.21 KB)

1004-file1.pdf (630.79 KB)

1004-file3.mp4 (21.42 MB)

Date

2019

Authors

Hamid, Sagad
Derstroff, Adrian
Klemm, Sören
Ngo, Quynh Quang
Jiang, Xiaoyi
Linsen, Lars

Publisher

The Eurographics Association

Abstract

A good deep neural network design allows for efficient training and high accuracy. The training step requires a suitable choice of several hyper-parameters. Limited knowledge exists on how the hyper-parameters impact the training process, what is the interplay of multiple hyper-parameters, and what is the interrelation of hyper-parameters and network topology. In this paper, we present a structured analysis towards these goals by investigating an ensemble of training runs.We propose a visual ensemble analysis based on hyper-parameter space visualizations, performance visualizations, and visualizing correlations of topological structures. As a proof of concept, we apply our approach to deep convolutional neural networks.

        @inproceedings{10.2312:mlvis.20191160
,
booktitle = {Machine Learning Methods in Visualisation for Big Data
},
editor = {Archambault, Daniel and Nabney, Ian and Peltonen, Jaakko
},
title = {{Visual Ensemble Analysis to Study the Influence of Hyper-parameters on Training Deep Neural Networks
}},
author = {Hamid, Sagad and 
Derstroff, Adrian and 
Klemm, Sören and 
Ngo, Quynh Quang and 
Jiang, Xiaoyi and 
Linsen, Lars
},
year = {2019
},
publisher = {The Eurographics Association
},
ISBN = {978-3-03868-089-5
},
DOI = {10.2312/mlvis.20191160
}
}