## Big Learning: A Universal Machine Learning Paradigm? A PyTorch Implementation

[//]: # (<p align="center">)

[//]: # (  <img src="" width="480">)

[//]: # (</p>)


This is a PyTorch/GPU implementation of the paper [Big Learning: A Universal Machine Learning Paradigm?](https://arxiv.org/pdf/2207.03899.pdf):

* On the text experiments associated with the GLUE benchmark.

* This repo is a modification on the [HuggingFace transformers repo](https://github.com/huggingface/transformers.git). Installation and preparation follow that repo.

### Catalog

- [x] Big learning on the RTE/MRPC/SST-2 GLUE tasks.




###  Big learning on the RTE task:

Based on the script ./examples/pytorch/text-classification/run_glue_biglearn.py


```
export TASK_NAME=rte

python run_glue_biglearn.py \
   --model_name_or_path xlnet-base-cased \
   --task_name $TASK_NAME \
   --do_train \
   --do_eval \
   --max_seq_length 128 \
   --per_device_train_batch_size 32 \
   --per_device_eval_batch_size 1 \
   --warmup_steps 120  \
   --learning_rate 6e-5 \
   --num_train_epochs 10 \
   --output_dir /tmp/$TASK_NAME/ \
   --overwrite_output_dir \
   --loss_betaBL 0.4
```
where task name can be one of rte, mrpc, sst2.

