论文标题
混合亚当和SGD：一种组合优化方法
Mixing ADAM and SGD: a Combined Optimization Method
论文作者
论文摘要
优化方法（优化器）在深度学习领域的有效培训有效地培训神经网络。在文献中，有许多论文比较了使用不同优化器训练的神经模型。每篇论文都表明，对于特定问题，优化器比其他问题更好，但是随着问题的变化，这种类型的结果不再有效，我们必须从头开始。在我们的论文中，我们建议使用两个非常不同的优化器的组合，但是当同时使用时，它们可以克服在非常不同的问题中的单个优化器的性能。我们提出了一个称为MAS（混合Adam和SGD）的新优化器，该优化器通过恒定重量的分配来同时权衡这两者的贡献，从而同时集成了SGD和Adam。而不是试图改善SGD或Adam，我们通过尽力而为同时利用这两者。我们使用各种CNN进行了几项有关图像和文本文档分类的实验，并且通过实验证明，所提出的MAS优化器比单个SGD或ADAM优化器产生的性能更好。源代码和实验的所有结果可在线可从以下链接获得https://gitlab.com/nicolalandro/multi \ _optimizer
Optimization methods (optimizers) get special attention for the efficient training of neural networks in the field of deep learning. In literature there are many papers that compare neural models trained with the use of different optimizers. Each paper demonstrates that for a particular problem an optimizer is better than the others but as the problem changes this type of result is no longer valid and we have to start from scratch. In our paper we propose to use the combination of two very different optimizers but when used simultaneously they can overcome the performances of the single optimizers in very different problems. We propose a new optimizer called MAS (Mixing ADAM and SGD) that integrates SGD and ADAM simultaneously by weighing the contributions of both through the assignment of constant weights. Rather than trying to improve SGD or ADAM we exploit both at the same time by taking the best of both. We have conducted several experiments on images and text document classification, using various CNNs, and we demonstrated by experiments that the proposed MAS optimizer produces better performance than the single SGD or ADAM optimizers. The source code and all the results of the experiments are available online at the following link https://gitlab.com/nicolalandro/multi\_optimizer
