论文标题
残留网络和非残余网络之间的插值
Interpolation between Residual and Non-Residual Networks
论文作者
论文摘要
尽管普通的微分方程（ODE）为设计网络体系结构提供了见解，但其与非残基卷积神经网络（CNN）的关系尚不清楚。在本文中，我们通过添加阻尼项提出了一种新颖的ODE模型。可以表明，提出的模型可以通过调整插值系数来恢复重新NET和CNN。因此，阻尼的ODE模型为解释残留和非残基网络提供了统一的框架。 Lyapunov分析揭示了所提出的模型的稳定性，从而可以提高学习网络的稳健性。许多图像分类基准的实验表明，所提出的模型显着提高了从随机噪声和对抗性攻击方法的扰动输入上的重新连接的准确性。此外，损失景观分析证明了我们方法沿攻击方向的鲁棒性提高。
Although ordinary differential equations (ODEs) provide insights for designing network architectures, its relationship with the non-residual convolutional neural networks (CNNs) is still unclear. In this paper, we present a novel ODE model by adding a damping term. It can be shown that the proposed model can recover both a ResNet and a CNN by adjusting an interpolation coefficient. Therefore, the damped ODE model provides a unified framework for the interpretation of residual and non-residual networks. The Lyapunov analysis reveals better stability of the proposed model, and thus yields robustness improvement of the learned networks. Experiments on a number of image classification benchmarks show that the proposed model substantially improves the accuracy of ResNet and ResNeXt over the perturbed inputs from both stochastic noise and adversarial attack methods. Moreover, the loss landscape analysis demonstrates the improved robustness of our method along the attack direction.
