Paper list This is the original full reading list that we used to select the papers we are reading in the class. Title Approach Domain Semi-Supervised Learning with Deep Generative Models Semi-Supervised Learning CV Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks Transfer Learning CV How transferable are features in deep neural networks? Transfer Learning CV Skip-Thought Vectors Transfer/Representation Learning NLP Semi-Supervised Sequence Learning Transfer Learning NLP Learning Distributed Representations of Sentences from Unlabelled Data Transfer Learning NLP Adversarial Training Methods for Semi-Supervised Text Classification Semi-Supervised Learning NLP Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning Semi-Supervised Learning CV What makes ImageNet good for transfer learning? Transfer Learning CV Temporal Ensembling for Semi-Supervised Learning Semi-Supervised Learning CV Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning Semi-Supervised Learning CV Supervised Learning of Universal Sentence Representations from Natural Language Inference Data Transfer/Representation Learning NLP Good Semi-supervised Learning that Requires a Bad GAN Semi-Supervised Learning CV Snorkel: Rapid Training Data Creation with Weak Supervision Weak/Active Learning Many Universal Language Model Fine-tuning for Text Classification Transfer Learning NLP Deep Contextualized Word Representations Transfer Learning NLP An efficient framework for learning sentence representations Transfer/Representation Learning NLP Universal Sentence Encoder Transfer/Representation Learning NLP Exploring the Limits of Weakly Supervised Pretraining Transfer Learning CV Do Better ImageNet Models Transfer Better? Transfer Learning CV Improving Language Understanding by Generative Pre-Training Transfer Learning NLP Understanding Back-Translation at Scale Semi-Supervised Learning NLP BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Transfer Learning NLP Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks Transfer Learning NLP Rethinking ImageNet Pre-Training Transfer Learning CV Cross-lingual Language Model Pretraining Transfer Learning NLP Multi-Task Deep Neural Networks for Natural Language Understanding Transfer Learning NLP Parameter-Efficient Transfer Learning for NLP Transfer Learning NLP Task2Vec: Task Embedding for Meta-Learning Transfer Learning CV To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks Transfer Learning NLP Unsupervised Data Augmentation for Consistency Training Semi-Supervised Learning CV + NLP Billion-scale semi-supervised learning for image classification Semi-Supervised Learning CV MixMatch: A Holistic Approach to Semi-Supervised Learning Semi-Supervised Learning CV MASS: Masked Sequence to Sequence Pre-training for Language Generation Transfer Learning NLP Unified Language Model Pre-training for Natural Language Understanding and Generation Transfer Learning NLP S4L: Self-Supervised Semi-Supervised Learning Semi-Supervised Learning CV How to Fine-Tune BERT for Text Classification? Transfer Learning NLP Data-Efficient Image Recognition with Contrastive Predictive Coding Transfer/Representation Learning CV Synthetic QA Corpora Generation with Roundtrip Consistency Semi-Supervised Learning NLP XLNet: Generalized Autoregressive Pretraining for Language Understanding Transfer Learning NLP Large Scale Adversarial Representation Learning Transfer/Representation Learning CV RoBERTa: A Robustly Optimized BERT Pretraining Approach Transfer Learning NLP Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning Semi-Supervised Learning CV ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Transfer Learning NLP Self-training with Noisy Student improves ImageNet classification Semi-Supervised Learning CV Momentum Contrast for Unsupervised Visual Representation Learning Transfer/Representation Learning CV Big Transfer (BiT): General Visual Representation Learning Transfer Learning CV Semi-Supervised Learning with Normalizing Flows Semi-Supervised Learning Many FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence Semi-Supervised Learning CV A Simple Framework for Contrastive Learning of Visual Representations Transfer/Representation Learning CV Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping Transfer Learning NLP ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators Transfer Learning NLP Rethinking Pre-training and Self-training Transfer and Semi-Supervised Learning CV Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning Transfer/Representation Learning CV Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Transfer/Representation Learning CV Big Self-Supervised Models are Strong Semi-Supervised Learners Semi-Supervised Learning CV