Scipediacontent: Scipediacontent moved page Draft Content 237076649 to Peng et al 2018a

2021-02-01T18:27:55Z

Scipediacontent moved page Draft Content 237076649 to Peng et al 2018a

Scipediacontent: Created page with " == Abstract == We introduce the structured projection of intermediate gradients optimization technique (SPIGOT), a new method for backpropagating through neural networks tha..."

2021-02-01T18:27:50Z

Created page with " == Abstract == We introduce the structured projection of intermediate gradients optimization technique (SPIGOT), a new method for backpropagating through neural networks tha..."

New page

== Abstract ==

We introduce the structured projection of intermediate gradients optimization technique (SPIGOT), a new method for backpropagating through neural networks that include hard-decision structured predictions (e.g., parsing) in intermediate layers. SPIGOT requires no marginal inference, unlike structured attention networks (Kim et al., 2017) and some reinforcement learning-inspired solutions (Yogatama et al., 2017). Like so-called straight-through estimators (Hinton, 2012), SPIGOT defines gradient-like quantities associated with intermediate nondifferentiable operations, allowing backpropagation before and after them; SPIGOT's proxy aims to ensure that, after a parameter update, the intermediate structure will remain well-formed. We experiment on two structured NLP pipelines: syntactic-then-semantic dependency parsing, and semantic parsing followed by sentiment classification. We show that training with SPIGOT leads to a larger improvement on the downstream task than a modularly-trained pipeline, the straight-through estimator, and structured attention, reaching a new state of the art on semantic dependency parsing.

Comment: ACL 2018

== Original document ==

The different versions of the original document can be found in:

* [http://arxiv.org/abs/1805.04658 http://arxiv.org/abs/1805.04658] under the license https://creativecommons.org/licenses/by

* [http://dx.doi.org/10.18653/v1/p18-1173 http://dx.doi.org/10.18653/v1/p18-1173]

* [https://www.aclweb.org/anthology/P18-1173.pdf https://www.aclweb.org/anthology/P18-1173.pdf]

* [https://dblp.uni-trier.de/db/journals/corr/corr1805.html#abs-1805-04658 https://dblp.uni-trier.de/db/journals/corr/corr1805.html#abs-1805-04658],
: [https://arxiv.org/abs/1805.04658 https://arxiv.org/abs/1805.04658],
: [https://www.aclweb.org/anthology/P18-1173 https://www.aclweb.org/anthology/P18-1173],
: [https://www.aclweb.org/anthology/P18-1173.pdf https://www.aclweb.org/anthology/P18-1173.pdf],
: [https://arxiv.org/pdf/1805.04658.pdf https://arxiv.org/pdf/1805.04658.pdf],
: [https://aclanthology.info/papers/P18-1173/p18-1173 https://aclanthology.info/papers/P18-1173/p18-1173],
: [https://academic.microsoft.com/#/detail/2964263959 https://academic.microsoft.com/#/detail/2964263959] under the license cc-by

← Older revision	Revision as of 18:27, 1 February 2021
(No difference)

Peng et al 2018a - Revision history

Scipediacontent: Scipediacontent moved page Draft Content 237076649 to Peng et al 2018a

Scipediacontent: Created page with " == Abstract == We introduce the structured projection of intermediate gradients optimization technique (SPIGOT), a new method for backpropagating through neural networks tha..."