Transformers

Attention From Scratch: How Transformers Replaced a Bottleneck with a Lookup

• Lakshmanan LN
• Machine learning, Deep learning, Nlp

Take a vanilla encoder-decoder RNN doing machine translation. It reads a source sentence token by token, updating one hidden-state vector as …