论文标题
在较重的尾巴下提高了可伸缩性，没有强大的凸度
Improved scalability under heavy tails, without strong convexity
论文作者
论文摘要
现实世界中的数据载有外围值。机器学习面临的挑战在于，学习者通常不知道其收到的反馈（损失，梯度等）是否会被重尾。在这项工作中，我们研究了一种简单的算法策略，当损失和梯度都可以重新尾部时，可以利用它。核心技术引入了一个简单的鲁棒验证子例程，该验证用于提高廉价基于梯度的子过程的置信度。与文献中最近强大的梯度下降方法相比，维度依赖性（风险范围和成本）大大改善，而无需依赖强凸度或每步昂贵的每步鲁棒化。从经验上讲，我们还表明，在重尾损失下，所提出的程序不能简单地被幼稚的交叉验证所取代。综上所述，我们有一种具有透明保证的可扩展方法，该方法的性能很好，而没有事先了解其收到的反馈的“方便”。
Real-world data is laden with outlying values. The challenge for machine learning is that the learner typically has no prior knowledge of whether the feedback it receives (losses, gradients, etc.) will be heavy-tailed or not. In this work, we study a simple algorithmic strategy that can be leveraged when both losses and gradients can be heavy-tailed. The core technique introduces a simple robust validation sub-routine, which is used to boost the confidence of inexpensive gradient-based sub-processes. Compared with recent robust gradient descent methods from the literature, dimension dependence (both risk bounds and cost) is substantially improved, without relying upon strong convexity or expensive per-step robustification. Empirically, we also show that under heavy-tailed losses, the proposed procedure cannot simply be replaced with naive cross-validation. Taken together, we have a scalable method with transparent guarantees, which performs well without prior knowledge of how "convenient" the feedback it receives will be.
