论文标题
通过逐步搜索张量环网络选择启发式排名
Heuristic Rank Selection with Progressively Searching Tensor Ring Network
论文作者
论文摘要
最近，张量环网络（TRN）已应用于深网络，在压缩比和准确性方面取得了显着的成功。尽管与TRN的性能高度相关，但在以前的作品中很少研究等级选择，并且通常在实验中设定为平等。同时，没有任何启发式方法可以选择排名，而找到适当等级的一种列举方法非常耗时。有趣的是，我们发现，等级要素的一部分是敏感的，通常在狭窄的区域（即兴趣区域）聚集。因此，基于上述现象，我们提出了一种逐渐搜索张量环网络搜索（PSTRN）的新型进行性遗传算法，该算法具有精确有效的最佳等级。通过进化阶段和渐进阶段，PSTRN可以快速收敛到兴趣区域并收获良好的性能。实验结果表明，与列举方法相比，PSTRN可以显着降低寻求等级的复杂性。此外，我们的方法在MNIST，CIFAR10/100，UCF11和HMDB51等公共基准上得到了验证，并实现了最先进的性能。
Recently, Tensor Ring Networks (TRNs) have been applied in deep networks, achieving remarkable successes in compression ratio and accuracy. Although highly related to the performance of TRNs, rank selection is seldom studied in previous works and usually set to equal in experiments. Meanwhile, there is not any heuristic method to choose the rank, and an enumerating way to find appropriate rank is extremely time-consuming. Interestingly, we discover that part of the rank elements is sensitive and usually aggregate in a narrow region, namely an interest region. Therefore, based on the above phenomenon, we propose a novel progressive genetic algorithm named Progressively Searching Tensor Ring Network Search (PSTRN), which has the ability to find optimal rank precisely and efficiently. Through the evolutionary phase and progressive phase, PSTRN can converge to the interest region quickly and harvest good performance. Experimental results show that PSTRN can significantly reduce the complexity of seeking rank, compared with the enumerating method. Furthermore, our method is validated on public benchmarks like MNIST, CIFAR10/100, UCF11 and HMDB51, achieving the state-of-the-art performance.
