nlp-tutorial-tensorflow2.x

​

nlp-tutorial-tensorflow2.x is a tutorial for who is studying NLP using TensorFlow2.x. Most of the model in NLP were implemented with less 100 lines of code(except comments or blank lines)

Reference: https://github.com/graykode/nlp-tutorial （implemented by Pytorch）

Curriculum - (Example Purpose)

1. Basic Embedding Model

• 1-1.NNLM(Neural Network Language Model) - Predict Next Word
  • Paper - A Neural Probabilistic Language Model(2003)
  • Code - NNLM.py
• 1-2.Word2Vec(Skip-gram) - Embedding Words and Show Graph
  • Paper - Distributed Representations of Words and Phrases and their Compositionality(2013)
  • Code - Word2Vec-Skipgram(Softmax).py
• 1-3.FastText(Application Level) - Sentence Classification
  • Paper - Bag of Tricks for Efficient Text Classification(2016)
  • Code - FastText.py

2. CNN(Convolutional Neural Network)

• 2-1.TextCNN - Binary Sentiment Classification
  • Paper - Convolutional Neural Networks for Sentence Classification(2014)
  • Code - TextCNN.py

3. RNN(Recurrent Neural Network)

• 3-1.TextRNN - Sentiment Classification
  • Paper - Finding Structure in Time(1990)
  • Code - TextRNN.py
• 3-2.TextLSTM - Autocomplete
  • Paper - LONG SHORT-TERM MEMORY(1997)
  • Code - TextLSTM.py
• 3-3.Bi-LSTM - Sequence Tagging
  • Code - Bi-LSTM.py

4. Attention Mechanism

• 4-1. Seq2Seq - Change Word
  • Paper - Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation(2014)
  • Code - Seq2Seq.py
• 4-2.Seq2Seq with Attention - Translate
  • Paper - Neural Machine Translation by Jointly Learning to Align and Translate(2014)
  • Code - Seq2Seq_attention.py
• 4-3.Bi-LSTM with Attention - Binary Sentiment Classification
  • Paper - Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification
  • Code - BiLSTM_attention.py

5. Model based on Transformer

• 5-1.The Transformer - Translate
  • Paper - Attention Is All You Need(2017)
  • Code - transformer.py

Dependencies

• Python 3.7+
• TensorFlow 2.3.0