|
DL之Encoder-Decoder：Encoder-Decoder模型的相關(guān)論文、設(shè)計(jì)思路、關(guān)鍵步驟等配圖集合之詳細(xì)攻略 Encoder-Decoder模型的相關(guān)論文1、Encoder-Decoder 結(jié)構(gòu)做機(jī)器翻譯任務(wù)的更多細(xì)節(jié)，可以參考 原始論文《Learning Phrase Representations using RNN Encoder– Decoder for Statistical Machine Translation》 Encoder-Decoder模型的設(shè)計(jì)思路Abstract：In this paper, we propose a novel neural network model called RNN Encoder– Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixedlength vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder–Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases. 1、An illustration of the proposed RNN Encoder–Decoder. 2、An illustration of the proposed hidden activation function. The update gate z selects whether the hidden state is to be updated with a new hidden state h?. The reset gate r decides whether the previous hidden state is ignored. See Eqs. (5)–(8) for the detailed equations of r, z, h and h?. 3、: BLEU scores computed on the development and test sets using different combinations of approaches. WP denotes a word penalty, where we penalizes the number of unknown words to neural networks. 4、2–D embedding of the learned word representation. The left one shows the full embedding space, while the right one shows a zoomed-in view of one region (color–coded). For more plots, see the supplementary material. 5、2–D embedding of the learned phrase representation. The top left one shows the full representation space (5000 randomly selected points), while the other three figures show the zoomed-in view of specific regions (color–coded). Encoder-Decoder模型的關(guān)鍵步驟1、E-D整體結(jié)構(gòu) 2、E-D步驟解釋 |
|
|
