Scipediacontent: Scipediacontent moved page Draft Content 567780869 to Zaharia et al 2018a

2021-02-01T23:17:00Z

Scipediacontent moved page Draft Content 567780869 to Zaharia et al 2018a

Scipediacontent: Created page with " == Abstract == Model diagnosis is the process of analyzing machine learning (ML) model performance to identify where the model works well and where it doesn’t. It is a key..."

2021-02-01T23:16:54Z

Created page with " == Abstract == Model diagnosis is the process of analyzing machine learning (ML) model performance to identify where the model works well and where it doesn’t. It is a key..."

New page

== Abstract ==

Model diagnosis is the process of analyzing machine learning (ML) model performance to identify where the model works well and where it doesn’t. It is a key part of the modeling process and helps ML developers iteratively improve model accuracy. Often, model diagnosis is performed by analyzing different datasets or inter- mediates associated with the model such as the input data and hidden representations learned by the model (e.g., [ 4 , 24 , 39 ]). The bottleneck in fast model diagnosis is the creation and storage of model intermediates. Storing these intermediates requires tens to hundreds of GB of storage whereas re-running the model for each diagnostic query slows down model diagnosis. To address this bottleneck, we propose a system called MISTIQUE that can work with traditional ML pipelines as well as deep neural networks to efficiently capture, store, and query model intermediates for diag- nosis. For each diagnostic query, MISTIQUE intelligently chooses whether to re-run the model or read a previously stored intermediate. For intermediates that are stored in MISTIQUE , we propose a range of optimizations to reduce storage footprint including quantization, summarization, and data deduplication. We evaluate our techniques on a range of real-world ML models in scikit-learn and Tensorflow. We demonstrate that our optimizations reduce storage by up to 110X for traditional ML pipelines and up to 6X for deep neural networks. Furthermore, by using MISTIQUE , we can speed up diagnostic queries on traditional ML pipelines by up to 390X and 210X on deep neural networks.

Facebook PhD Fellowship

Alfred P. Sloan Foundation. University Centers for Exemplary Mentoring (UCEM) fellowshi

Document type: Conference object

== Full document ==
<pdf>Media:Draft_Content_567780869-beopen3224-2330-document.pdf</pdf>

== Original document ==

The different versions of the original document can be found in:

* [https://dspace.mit.edu/bitstream/1721.1/121346/2/sigmod_mistique.pdf https://dspace.mit.edu/bitstream/1721.1/121346/2/sigmod_mistique.pdf] under the license https://creativecommons.org/licenses/by-nc-sa

* [https://hdl.handle.net/1721.1/121346 https://hdl.handle.net/1721.1/121346] under the license cc-by-nc-sa

* [http://dl.acm.org/ft_gateway.cfm?id=3196934&ftid=1976697&dwn=1 http://dl.acm.org/ft_gateway.cfm?id=3196934&ftid=1976697&dwn=1],
: [http://dx.doi.org/10.1145/3183713.3196934 http://dx.doi.org/10.1145/3183713.3196934] under the license http://creativecommons.org/licenses/by-nc-sa/4.0/

* [https://dblp.uni-trier.de/db/conf/sigmod/sigmod2018.html#VartakTMZ18 https://dblp.uni-trier.de/db/conf/sigmod/sigmod2018.html#VartakTMZ18],
: [https://doi.acm.org/10.1145/3183713.3196934 https://doi.acm.org/10.1145/3183713.3196934],
: [https://dl.acm.org/citation.cfm?id=3196934 https://dl.acm.org/citation.cfm?id=3196934],
: [https://dl.acm.org/citation.cfm?doid=3183713.3196934 https://dl.acm.org/citation.cfm?doid=3183713.3196934],
: [https://doi.org/10.1145/3183713.3196934 https://doi.org/10.1145/3183713.3196934],
: [https://academic.microsoft.com/#/detail/2798535736 https://academic.microsoft.com/#/detail/2798535736] under the license http://www.acm.org/publications/policies/copyright_policy#Background

← Older revision	Revision as of 23:17, 1 February 2021
(No difference)

Zaharia et al 2018a - Revision history

Scipediacontent: Scipediacontent moved page Draft Content 567780869 to Zaharia et al 2018a

Scipediacontent: Created page with " == Abstract == Model diagnosis is the process of analyzing machine learning (ML) model performance to identify where the model works well and where it doesn’t. It is a key..."