Skip to content

CopyNet (Copy Mechanism in Seq2Seq) implementation with TensorFlow 2

Notifications You must be signed in to change notification settings

pavanchhatpar/copynet-tf

Repository files navigation

CopyNet implementation with TensorFlow 2

  • Incorporating Copying Mechanism in Sequence-to-Sequence Learning
  • Uses TensorFlow 2.0 and above APIs with tf.keras too
  • Adapted from AllenNLP's PyTorch implementation, their blog referenced below was very helpful to understand the math from an implementation perspective

Python package Upload Python Package

Install package

Using pip

pip install copynet-tf

Compile from source

python -m pip install --upgrade pip
pip install setuptools wheel
python setup.py sdist bdist_wheel

Examples

Abstract usage

...
import MyEncoder
from copynet_tf import GRUDecoder

...

class MyModel(tf.keras.Model):

  ...

  def call(self, X, y, training):
    source_token_ids, source2target_ids = X
    target_token_ids, target2source_ids = y

    enc_output, enc_final_output, mask = self.encoder(X, y, training)
    output_dict = self.decoder(
      source_token_ids, source2target_ids, mask,
      target_token_ids, target2source_ids, training)
    return output_dict

  ...

Concrete examples

Find concrete examples inside examples folder of the repo

References

  • Incorporating Copying Mechanism in Sequence-to-Sequence Learning: (paper)
  • AllenNLP implementation: (blog) (code)
  • BLEU score metric: (code)