论文标题
可拉长的细胞帮助飞镖搜索更好
Stretchable Cells Help DARTS Search Better
论文作者
论文摘要
可区分的神经结构搜索（DARTS）在发现灵活和多样化的细胞类型方面取得了很大的成功。为了减少评估差距，预计超级网与目标网络具有相同的层。但是，即使对于这种一致的搜索，搜索的细胞也经常遭受性能较差，尤其是对于较少层的超级网络，因为当前的飞镖方法容易进入宽且浅的细胞，并且这种拓扑崩溃会导致亚最佳搜索细胞。在本文中，我们通过赋予细胞具有明确的可伸缩性来减轻此问题，因此可以同时在可拉伸单元上实现搜索，同时在可伸缩的单元格上实施。具体而言，我们引入了一组拓扑变量和组合概率分布，以明确对目标拓扑进行建模。我们的方法具有更多样化和复杂的拓扑结构，适应各种层数。关于CIFAR-10和Imagenet的广泛实验表明，我们的可拉伸细胞以更少的层和参数获得更好的性能。例如，我们的方法可以在CIFAR-10数据集上提高飞镖的0.28 \％准确性，而Imagenet数据集上具有45 \％参数的降低或2.9 \％。
Differentiable neural architecture search (DARTS) has gained much success in discovering flexible and diverse cell types. To reduce the evaluation gap, the supernet is expected to have identical layers with the target network. However, even for this consistent search, the searched cells often suffer from poor performance, especially for the supernet with fewer layers, as current DARTS methods are prone to wide and shallow cells, and this topology collapse induces sub-optimal searched cells. In this paper, we alleviate this issue by endowing the cells with explicit stretchability, so the search can be directly implemented on our stretchable cells for both operation type and topology simultaneously. Concretely, we introduce a set of topological variables and a combinatorial probabilistic distribution to explicitly model the target topology. With more diverse and complex topologies, our method adapts well for various layer numbers. Extensive experiments on CIFAR-10 and ImageNet show that our stretchable cells obtain better performance with fewer layers and parameters. For example, our method can improve DARTS by 0.28\% accuracy on CIFAR-10 dataset with 45\% parameters reduced or 2.9\% with similar FLOPs on ImageNet dataset.
