RNN Encoder-Decoder - created from Mind Map

Page 1

Encode a variable-length sequence into a fixed-length vector representation

Decode a given fixed-length vector representation back into a variable-length sequence

From a probabilistic perspective, this model is a general method to learn the conditional distribution over a variable-length sequence conditioned on yet another variable-length sequence: p(y1, . . . , yT' | x1, . . . , xT) T!=T'

The encoder is an RNN that reads each symbol of an input sequence x sequentially. As it reads each symbol, the hidden state of the RNN changes according to : h(t)=f(h(t-1),x_t). After reading the end of the sequence (marked by an end-of-sequence symbol), the hidden state of the RNN is a summary c of the whole input sequen

The decoder is another RNN which is trained to generate the output sequence by predicting the next symbol y_t given the hidden state h(t) = f(h(t-1), y_(t-1), c) is conditionned on previous h, y and c:the summary of the input.

Rnn Encoder Decoder Article (application/pdf)