Visual Analysis of the Impact of Neural Network Hyper-Parameters
Loading...
Date
2020
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
We present an analysis of the impact of hyper-parameters for an ensemble of neural networks using tailored visualization techniques to understand the complicated relationship between hyper-parameters and model performance. The high-dimensional error surface spanned by the wide range of hyper-parameters used to specify and optimize neural networks is difficult to characterize - it is non-convex and discontinuous, and there could be complex local dependencies between hyper-parameters. To explore these dependencies, we make use of a large number of sampled relations between hyper-parameters and end performance, retrieved from thousands of individually trained convolutional neural network classifiers. We use a structured selection of visualization techniques to analyze the impact of different combinations of hyper-parameters. The results reveal how complicated dependencies between hyper-parameters influence the end performance, demonstrating how the complete picture painted by considering a large number of trainings simultaneously can aid in understanding the impact of hyper-parameter combinations.
Description
@inproceedings{10.2312:mlvis.20201101,
booktitle = {Machine Learning Methods in Visualisation for Big Data},
editor = {Archambault, Daniel and Nabney, Ian and Peltonen, Jaakko},
title = {{Visual Analysis of the Impact of Neural Network Hyper-Parameters}},
author = {Jönsson, Daniel and Eilertsen, Gabriel and Shi, Hezi and Zheng, Jianmin and Ynnerman, Anders and Unger, Jonas},
year = {2020},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-113-7},
DOI = {10.2312/mlvis.20201101}
}