13. Visual Representation Learning

What are good representations?

Supervised learning & fine-tuning

• Transfer learning
• Fine-tune in practice

Unsupervised learning

• Auto-encoder
• Data-compression
• Distribution modeling, EBM

Self-supervised learning

• Predictive learning
• SimCLR
• Im2vec, word2vec
• Context prediction
• Cross-modal self-supervised learning

Discrete & sequence learning

• Discretized visual representation
• Visual sequence learning for non-sequential data
• Gato, Unified IO