论文标题
renfeation：一种简单的转移学习方法，可改善对抗性鲁棒性
Renofeation: A Simple Transfer Learning Method for Improved Adversarial Robustness
论文作者
论文摘要
通过从大规模数据集上的预训练模型转移的知识转移是一种广泛扩展的方法，可以有效地在小规模数据集上构建模型。在这项工作中，我们表明，最近旨在通过重新训练进行转移学习的对抗性攻击可以成功地欺骗通过端到端微调通过转移学习训练的模型。这引起了许多工业应用的安全问题。相比之下，尽管这些模型的精度通常更低，但对这种攻击进行了随机初始化训练的模型对此类攻击更为强大。为此，我们提出了一种嘈杂的特征蒸馏，这是一种新的转移学习方法，该方法从随机初始化中训练网络，同时通过微调实现清洁数据表现竞争力。代码可在https://github.com/cmu-enyac/renofeation上找到。
Fine-tuning through knowledge transfer from a pre-trained model on a large-scale dataset is a widely spread approach to effectively build models on small-scale datasets. In this work, we show that a recent adversarial attack designed for transfer learning via re-training the last linear layer can successfully deceive models trained with transfer learning via end-to-end fine-tuning. This raises security concerns for many industrial applications. In contrast, models trained with random initialization without transfer are much more robust to such attacks, although these models often exhibit much lower accuracy. To this end, we propose noisy feature distillation, a new transfer learning method that trains a network from random initialization while achieving clean-data performance competitive with fine-tuning. Code available at https://github.com/cmu-enyac/Renofeation.
