论文标题
通过共轭矩阵合奏在深神经网络中的等效性
Equivalence in Deep Neural Networks via Conjugate Matrix Ensembles
论文作者
论文摘要
开发了一种用于检测深度学习体系结构的等效性的数值方法。该方法基于从深神经网络重量矩阵中生成混合矩阵集合（MME），而{\ IT结合圆形集合}与神经体系结构拓扑匹配。此后，经验证据支持{\ IT现象}，即神经结构的光谱密度与相应的{\ IT结合圆形合奏}之间的差异在频谱的长正尾部的不同衰减速率上消失了，即累积的圆形光谱差异（CSD）。通过分析CSD的波动，可以将这一发现用于在不同神经体系结构之间建立等效。我们研究了这种现象，用于广泛的深度学习视觉架构，并以圆形合奏起源于统计量子力学。讨论了对人工和自然神经体系结构的拟议方法的实际含义，例如在神经结构搜索（NAS）中使用该方法的可能性和生物神经网络的分类。
A numerical approach is developed for detecting the equivalence of deep learning architectures. The method is based on generating Mixed Matrix Ensembles (MMEs) out of deep neural network weight matrices and {\it conjugate circular ensemble} matching the neural architecture topology. Following this, the empirical evidence supports the {\it phenomenon} that difference between spectral densities of neural architectures and corresponding {\it conjugate circular ensemble} are vanishing with different decay rates at the long positive tail part of the spectrum i.e., cumulative Circular Spectral Difference (CSD). This finding can be used in establishing equivalences among different neural architectures via analysis of fluctuations in CSD. We investigated this phenomenon for a wide range of deep learning vision architectures and with circular ensembles originating from statistical quantum mechanics. Practical implications of the proposed method for artificial and natural neural architectures discussed such as the possibility of using the approach in Neural Architecture Search (NAS) and classification of biological neural networks.
