RiskFix: Supporting Expert Validation of Predictive Timeseries Models in High-Intensity Settings
Loading...
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Many real-world machine learning workflows exist in longitudinal, interactive machine learning (ML) settings. This longitudinal nature is often due to incremental increasing of data, e.g., in clinical settings, where observations about patients evolve over their care period. Additionally, experts may become a bottleneck in the workflow, as their limited availability, combined with their role as human oracles, often leads to a lack of ground truth data. In such cases where ground truth data is small, the validation of interactive machine learning workflows relies on domain experts. Only those humans can assess the validity of a model prediction, especially in new situations that have been covered only weakly by available training data. Based on our experiences working with domain experts of a pediatric hospital's intensive care unit, we derive requirements for the design of support interfaces for the validation of interactive ML workflows in fast-paced, high-intensity environments. We present RiskFix, a software package optimized for the validation workflow of domain experts of such contexts. RiskFix is adapted to the cognitive resources and needs of domain experts in validating and giving feedback to the model. Also, RiskFix supports data scientists in their model-building work, with appropriate data structuring for the re-calibration (and possible retraining) of ML models.
Description
CCS Concepts: Computing methodologies -> Model verification and validation; Human-centered computing -> Open source software; Applied computing -> Health care information systems
@inproceedings{10.2312:evs.20231036,
booktitle = {EuroVis 2023 - Short Papers},
editor = {Hoellt, Thomas and Aigner, Wolfgang and Wang, Bei},
title = {{RiskFix: Supporting Expert Validation of Predictive Timeseries Models in High-Intensity Settings}},
author = {Morgenshtern, Gabriela and Verma, Arnav and Tonekaboni, Sana and Greer, Robert and Bernard, Jürgen and Mazwi, Mjaye and Goldenberg, Anna and Chevalier, Fanny},
year = {2023},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-219-6},
DOI = {10.2312/evs.20231036}
}