# Machine translation experiments
This is a guide on how to reproduce model trainings for machine translation with **relaxed attention** based on the cutoff augmentation method

* [*A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation*](https://arxiv.org/abs/2009.13818)



## Installation

First, create a conda virtual environment and activate it:
```
conda create -n fairseq_env python=3.6 -y
conda activate fairseq_env
```

Then, install fairseq and the other packages:
```
cd fairseq
pip install --editable ./
```
Also, install NVIDIA's apex extension for faster mixed precision  training:
``` bash
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" \
  --global-option="--deprecated_fused_adam" --global-option="--xentropy" \
  --global-option="--fast_multihead_attn" ./
cd ..
```



## Data preparation & Training

Follow the steps in [`fairseq/run.sh`](fairseq/run.sh) to pre-process the data necessary for fine-tuning, training and evaluation.


Relevant parameters for relaxed attention are:
```
# relaxed attention related parameters
relaxAttn=0.2
relaxSelfAttn=0.01
relaxation_matched_inference=false
attention_sigmoid_smoothing=false
```

