论文标题
IPMS分配鲁棒性以及与正则化和gan的链接
Distributional Robustness with IPMs and links to Regularization and GANs
论文作者
论文摘要
由于深度神经网络对小扰动的脆弱性，对对抗性攻击的鲁棒性是一个重要的问题，并且近年来引起了很多关注。分布强劲的优化（DRO）是解决这一挑战的一种特别有希望的方法，它通过基于差异的不确定性集来研究鲁棒性，并为诸如正则化之类的鲁棒化策略提供了宝贵的见解。在机器学习的背景下，大多数现有结果选择了$ f $ divergences，Wasserstein距离以及最近，最大平均差异（MMD）来构建不确定性集。我们通过研究用积分概率指标（IPMS）构建的不确定性集（包括MMD，MMD，Total Attrot Wariation and Wasserstein距离）来扩展这一工作的目的，以通过正规化来理解鲁棒性。我们的主要结果表明，在\ textit {任何}选择IPM下的DRO对应于正规化惩罚家族，该家族在MMD和Wasserstein距离的情况下恢复并改善了现有结果。由于我们的结果的普遍性，我们表明IPM的其他选择与机器学习中其他常用的惩罚相对应。此外，我们将结果扩展到通过$ f $ gans的对抗生成建模的揭示，这构成了第一个针对$ f $ gan目标的分配鲁棒性研究。我们的结果揭示了与鲁棒性有关的歧视者的归纳特性，使我们能够对几种基于惩罚的GAN方法（例如Wasserstein-，MMD-和Sobolev-Gans）发表积极评论。总而言之，我们的结果将gans与分布鲁棒性紧密联系在一起，扩展了对DRO的先前结果，并有助于我们理解正规化与鲁棒性之间的联系。
Robustness to adversarial attacks is an important concern due to the fragility of deep neural networks to small perturbations and has received an abundance of attention in recent years. Distributionally Robust Optimization (DRO), a particularly promising way of addressing this challenge, studies robustness via divergence-based uncertainty sets and has provided valuable insights into robustification strategies such as regularization. In the context of machine learning, the majority of existing results have chosen $f$-divergences, Wasserstein distances and more recently, the Maximum Mean Discrepancy (MMD) to construct uncertainty sets. We extend this line of work for the purposes of understanding robustness via regularization by studying uncertainty sets constructed with Integral Probability Metrics (IPMs) - a large family of divergences including the MMD, Total Variation and Wasserstein distances. Our main result shows that DRO under \textit{any} choice of IPM corresponds to a family of regularization penalties, which recover and improve upon existing results in the setting of MMD and Wasserstein distances. Due to the generality of our result, we show that other choices of IPMs correspond to other commonly used penalties in machine learning. Furthermore, we extend our results to shed light on adversarial generative modelling via $f$-GANs, constituting the first study of distributional robustness for the $f$-GAN objective. Our results unveil the inductive properties of the discriminator set with regards to robustness, allowing us to give positive comments for several penalty-based GAN methods such as Wasserstein-, MMD- and Sobolev-GANs. In summary, our results intimately link GANs to distributional robustness, extend previous results on DRO and contribute to our understanding of the link between regularization and robustness at large.
