Scipediacontent: Scipediacontent moved page Draft Content 453611053 to Lee et al 2020a

2021-01-28T23:13:50Z

Scipediacontent moved page Draft Content 453611053 to Lee et al 2020a

Scipediacontent: Created page with " == Abstract == We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulat..."

2021-01-28T23:13:44Z

Created page with " == Abstract == We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulat..."

New page

== Abstract ==

We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulation of the Multivariate Normal Distribution (MND), also known as the information form. The key insight of our work is that the information matrix, i.e. the inverse of the covariance matrix tends to be sparse in its spectrum. Therefore, dimensionality reduction techniques such as low rank approximations (LRA) can be effectively exploited. To achieve this, we develop a novel sparsification algorithm and derive a cost-effective analytical sampler. As a result, we show that the information form can be scalably applied to represent model uncertainty in DNNs. Our exhaustive theoretical analysis and empirical evaluations on various benchmarks show the competitiveness of our approach over the current methods.

Comment: Accepted to the Thirty-seventh International Conference on Machine Learning (ICML) 2020

== Original document ==

The different versions of the original document can be found in:

* [http://arxiv.org/abs/2006.11631 http://arxiv.org/abs/2006.11631]

* [https://elib.dlr.de/135531 https://elib.dlr.de/135531]

← Older revision	Revision as of 23:13, 28 January 2021
(No difference)

Lee et al 2020a - Revision history

Scipediacontent: Scipediacontent moved page Draft Content 453611053 to Lee et al 2020a

Scipediacontent: Created page with " == Abstract == We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulat..."