论文标题
通过遍历大地学来稀疏网络
Sparsifying networks by traversing Geodesics
论文作者
论文摘要
重量空间的几何形状和神经网络的功能流形在“理解” ML的复杂性方面起着重要作用。在本文中，我们试图通过通过几何形状的镜头查看ML中的某些开放问题，最终将其与在这些空间中的点或等效功能的发现或路径联系起来。我们提出了一个数学框架，以评估功能空间中的测量学，以找到从密集网络到其稀疏对应物的高性能路径。我们的结果是在接受CIFAR-10培训的VGG-11和MLP培训的MNIST上获得的。从广义上讲，我们证明该框架是一般的，可以应用于各种各样的问题，从稀疏到减轻灾难性遗忘。
The geometry of weight spaces and functional manifolds of neural networks play an important role towards 'understanding' the intricacies of ML. In this paper, we attempt to solve certain open questions in ML, by viewing them through the lens of geometry, ultimately relating it to the discovery of points or paths of equivalent function in these spaces. We propose a mathematical framework to evaluate geodesics in the functional space, to find high-performance paths from a dense network to its sparser counterpart. Our results are obtained on VGG-11 trained on CIFAR-10 and MLP's trained on MNIST. Broadly, we demonstrate that the framework is general, and can be applied to a wide variety of problems, ranging from sparsification to alleviating catastrophic forgetting.
