论文标题
稀疏神经网络的Pac-Bayes Oracle不平等
A PAC-Bayes oracle inequality for sparse neural networks
论文作者
论文摘要
我们研究了在非参数回归环境中稀疏深神经网的Gibbs后部分布。可以通过大都市调整后的兰格文算法访问后部。在稀疏的网络权重集上使用均匀先验的混合物，我们证明了一种甲骨文不等式，该不平等表明该方法适应了回归函数的未知规则性和分层结构。估计器达到了最小的收敛速率（最高为对数因子）。
We study the Gibbs posterior distribution for sparse deep neural nets in a nonparametric regression setting. The posterior can be accessed via Metropolis-adjusted Langevin algorithms. Using a mixture over uniform priors on sparse sets of network weights, we prove an oracle inequality which shows that the method adapts to the unknown regularity and hierarchical structure of the regression function. The estimator achieves the minimax-optimal rate of convergence (up to a logarithmic factor).
