Aug 12, 2020

GAN for abstractive summarisation:

• Finished reading paper: Higlights are uploaded here

Get To The Point - Summarization with Pointer-Generator Networks:

• Finished reading paper: Highlights are uploaded here
• Understood the pointer generator mechanism to be soft switch to select word token b/w source text attention and Pvoc.
• This gives the model edge as it can select OOV(Out of Vocabulary) word and also from vocabulary
• Coverage Mechanism: coverage vector ct, which is the sum of attention distributions over all previous decoder timesteps:
  • covlosst =∑imin(ati,cti)

General defination of Batch, Epoch, Iteration:

• Epoch: Number of time you go over the data set
• Batch: Consists of sample of you data for which you compute gradient and update weights:
  • Training on single batch involve forward pass + Backward pass
  • Forward Pass: Compute Loss
  • Backward Pass: Compute Gradient and Update weights
• Iteration: Completion of 1 batch == 1 iteration
  • Thus in 1 Epoch there will be |Trainin Data Size| / |Batch Sizee| number of iterations

Example: If I have training data (100000 samples) and I consider batch_size = 100, then train for 50 Epochs means I will have

Number of Iterations = 50 * (100000 / 100) = 50000 iterations.