论文标题
在随机子集概括误差边界和随机梯度Langevin Dynamics算法上
On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithm
论文作者
论文摘要
在这项工作中，我们使用Hellström和Durisi开发的框架[1]统一了基于随机子集的几个预期概括误差界限。首先，我们根据BU等人的各个样本互信息恢复界限。 [2]以及Negrea等人的数据集的随机子集。 [3]。然后，我们在Steinke和Zakynthinou [4]的随机子样本设置中介绍了它们的新的类似边界，并确定了框架的一些局限性。最后，我们扩展了Haghifam等人的边界。 [5]对于Langevin动力学到随机梯度Langevin动力学，我们将它们改进，以使其具有潜在的梯度规范的损失函数。
In this work, we unify several expected generalization error bounds based on random subsets using the framework developed by Hellström and Durisi [1]. First, we recover the bounds based on the individual sample mutual information from Bu et al. [2] and on a random subset of the dataset from Negrea et al. [3]. Then, we introduce their new, analogous bounds in the randomized subsample setting from Steinke and Zakynthinou [4], and we identify some limitations of the framework. Finally, we extend the bounds from Haghifam et al. [5] for Langevin dynamics to stochastic gradient Langevin dynamics and we refine them for loss functions with potentially large gradient norms.
