论文标题
对比度自我监督学习和设计新方法的框架
A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
论文作者
论文摘要
对比性自我监督学习（CSL）是一种通过求解从未标记的数据集中选择和比较锚点，负和正（APN）特征的借口任务来学习有用表示的方法。我们提出了一个概念框架，该框架表征了五个方面的CSL方法（1）数据增强管道，（2）编码器选择，（3）表示提取，（4）相似性度量和（5）损失函数。我们分析了三种领先的CSL方法-AMDIM，CPC和SIMCLR-，并表明，尽管有不同的动机，但在此框架下它们是特殊情况。我们通过设计另一个昏暗（Yadim）来展示框架的实用性，该型号（Yadim）在CIFAR-10，STL-10和Imagenet上取得了竞争性结果，并且更适合选择编码器和表示策略。为了支持正在进行的CSL研究，我们发布了该概念框架的Pytorch实施以及AMDIM，CPC（V2），SIMCLR，BYOL，MOCO，MOCO（V2）和YADIM的标准化实现。
Contrastive self-supervised learning (CSL) is an approach to learn useful representations by solving a pretext task that selects and compares anchor, negative and positive (APN) features from an unlabeled dataset. We present a conceptual framework that characterizes CSL approaches in five aspects (1) data augmentation pipeline, (2) encoder selection, (3) representation extraction, (4) similarity measure, and (5) loss function. We analyze three leading CSL approaches--AMDIM, CPC, and SimCLR--, and show that despite different motivations, they are special cases under this framework. We show the utility of our framework by designing Yet Another DIM (YADIM) which achieves competitive results on CIFAR-10, STL-10 and ImageNet, and is more robust to the choice of encoder and the representation extraction strategy. To support ongoing CSL research, we release the PyTorch implementation of this conceptual framework along with standardized implementations of AMDIM, CPC (V2), SimCLR, BYOL, Moco (V2) and YADIM.
