论文标题
思考全球，本地ACT：将DNN概括和节点级别的SNR关联
Think Global, Act Local: Relating DNN generalisation and node-level SNR
论文作者
论文摘要
良好的DNN概括背后的原因仍然是一个悬而未决的问题。在本文中，我们通过查看网络中节点的信噪比来探讨问题。从信息理论原理开始，可以得出DNN节点输出的SNR的表达式。使用此表达式，我们构建了标准图，以量化节点的权重优化SNR（或等效地，信息速率）。应用这些指标，我们举例说明促进良好SNR性能的重量集也表现出良好的概括。此外，我们能够确定表现出良好SNR行为的重量集的品质，从而促进良好的概括。这导致讨论这些结果如何与网络培训和正则化有关。最后，我们确定了可以在训练设计中使用这些观察结果的一些方法。
The reasons behind good DNN generalisation remain an open question. In this paper we explore the problem by looking at the Signal-to-Noise Ratio of nodes in the network. Starting from information theory principles, it is possible to derive an expression for the SNR of a DNN node output. Using this expression we construct figures-of-merit that quantify how well the weights of a node optimise SNR (or, equivalently, information rate). Applying these figures-of-merit, we give examples indicating that weight sets that promote good SNR performance also exhibit good generalisation. In addition, we are able to identify the qualities of weight sets that exhibit good SNR behaviour and hence promote good generalisation. This leads to a discussion of how these results relate to network training and regularisation. Finally, we identify some ways that these observations can be used in training design.
