Pattern-Exploiting Training for NLP!

This video explains Pattern-Exploiting Training (PET), a new technique to leverage pre-trained language models to label data for downstream tasks such as Text Classification or Natural Language Inference. This is done by introducing "patterns" or templates that guide the language model to label the review or pair of sentences. This also include a "verbalizer" that maps from the language model's vocabulary into downstream task labels! This tutorial also uses HuggingFace's NLP viewer and Renato Violin's Next Word Prediction Demo that I highly recommend checking out!

Thank you for watching! Please Subscribe!

Links:
Paper (Pattern-Exploiting Training): https://arxiv.org/pdf/2001.07676.pdf
HuggingFace NLP Viewer: https://huggingface.co/nlp/viewer/
Next-Word-Prediction: https://github.com/renatoviolin/next_...
PragmaticML blog summary of PET: https://www.pragmatic.ml/pet/
Self-Training with Noisy Student: https://arxiv.org/abs/1911.04252
FixMatch: https://arxiv.org/abs/2001.07685