Dynamic I/O Budget Reallocation For In Situ Wavelet Compression

Marsaglia, Nicole J.; Li, Shaomeng; Belcher, Kristi; Larsen, Matthew; Childs, Hank

Dynamic I/O Budget Reallocation For In Situ Wavelet Compression

Files

001-005.pdf (610.57 KB)

Date

2019

Authors

Marsaglia, Nicole J.
Li, Shaomeng
Belcher, Kristi
Larsen, Matthew
Childs, Hank

Publisher

The Eurographics Association

Abstract

In situ wavelet compression is a potential solution for enabling post hoc visualization on supercomputers with slow I/O systems. While this in situ compression is typically accomplished by allocating an equal storage budget to each parallel process, we propose an adaptive approach. With our approach, we introduce an assessment step prior to compression, where each process characterizes the variation in its portion of the data, and then dynamically adapts storage budgets to the processes with the most variation. We conducted experiments comparing our adaptive approach with the traditional, non-adaptive approach, on two different simulation codes with concurrencies of 512 cores and mesh resolutions of one billion cells. Our findings show that our adaptive approach yields three orders of magnitude of improvement for one simulation and is not harmful for the other.

        @inproceedings{10.2312:pgv.20191104
,
booktitle = {Eurographics Symposium on Parallel Graphics and Visualization
},
editor = {Childs, Hank and Frey, Steffen
},
title = {{Dynamic I/O Budget Reallocation For In Situ Wavelet Compression
}},
author = {Marsaglia, Nicole J. and 
Li, Shaomeng and 
Belcher, Kristi and 
Larsen, Matthew and 
Childs, Hank
},
year = {2019
},
publisher = {The Eurographics Association
},
ISSN = {1727-348X
},
ISBN = {978-3-03868-079-6
},
DOI = {10.2312/pgv.20191104
}
}