论文标题
解开神经建筑搜索
Disentangled Neural Architecture Search
论文作者
论文摘要
神经建筑搜索最近在各个领域都表现出了巨大的潜力。但是，现有方法在很大程度上依赖黑框控制器来搜索体系结构，这遭受了缺乏可解释性的严重问题。在本文中，我们提出了分离的神经体系结构搜索（DNA），该搜索将控制器的隐藏表示形式分解为语义上有意义的概念，从而可以解释神经体系结构搜索过程。基于系统的研究，我们发现网络体系结构与其性能之间的相关性，并提出了一个密集采样策略，以在有希望的地区进行有针对性的搜索，以产生表现良好的体系结构。我们表明：1）DNA成功地删除了体系结构表示形式，包括操作选择，跳过连接和层数。 2）从解释性中受益，DNA可以灵活地找到不同的拖船限制下的出色架构。 3）密集采样会导致神经架构搜索，并具有更高的效率和更好的性能。在NASBENCH-101数据集上，DNA使用基线方法的计算成本少于1/13，达到94.21％的最新性能。在Imagenet数据集上，DNA发现了达到22.7％测试错误的竞争架构。我们的方法提供了了解神经体系结构搜索的新观点。
Neural architecture search has shown its great potential in various areas recently. However, existing methods rely heavily on a black-box controller to search architectures, which suffers from the serious problem of lacking interpretability. In this paper, we propose disentangled neural architecture search (DNAS) which disentangles the hidden representation of the controller into semantically meaningful concepts, making the neural architecture search process interpretable. Based on systematical study, we discover the correlation between network architecture and its performance, and propose a dense-sampling strategy to conduct a targeted search in promising regions that may generate well-performing architectures. We show that: 1) DNAS successfully disentangles the architecture representations, including operation selection, skip connections, and number of layers. 2) Benefiting from interpretability, DNAS can find excellent architectures under different FLOPS restrictions flexibly. 3) Dense-sampling leads to neural architecture search with higher efficiency and better performance. On the NASBench-101 dataset, DNAS achieves state-of-the-art performance of 94.21% using less than 1/13 computational cost of baseline methods. On ImageNet dataset, DNAS discovers the competitive architectures that achieves 22.7% test error. our method provides a new perspective of understanding neural architecture search.
