论文标题
TF-NAS：重新考虑延迟约束的可区分神经体系结构搜索的三个搜索自由
TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search
论文作者
论文摘要
随着可区分的神经体系结构搜索（NAS）的蓬勃发展，自动搜索潜伏期约束的建筑为减少人工劳动和专业知识提供了新的观点。但是，搜索的架构通常在准确性上是最佳的，并且目标延迟周围可能会有巨大的烦恼。在本文中，我们重新考虑了三个可区分的NAS的自由，即操作级别，深度级别和宽度级别，并提出了一种新的方法，称为三列NAS（TF-NAS），以达到良好的分类准确性和精确的延迟约束。对于操作级别，我们提出一个双重采样搜索算法，以缓和操作崩溃。对于深度级别，我们引入了一个连接的搜索空间，以确保跳过和其他候选操作之间的相互排斥，并消除架构冗余。对于宽度级别，我们提出了一种弹性式策略，该策略以逐渐细粒的方式实现精确的延迟约束。对成像网的实验证明了TF-NAS的有效性。特别是，我们搜索的TF-NAS-A获得了76.9％的TOP-1准确性，以较小的延迟获得最新的结果。 1泰坦RTX GPU的总搜索时间仅为1.8天。代码可在https://github.com/aberhu/tf-nas上找到。
With the flourish of differentiable neural architecture search (NAS), automatically searching latency-constrained architectures gives a new perspective to reduce human labor and expertise. However, the searched architectures are usually suboptimal in accuracy and may have large jitters around the target latency. In this paper, we rethink three freedoms of differentiable NAS, i.e. operation-level, depth-level and width-level, and propose a novel method, named Three-Freedom NAS (TF-NAS), to achieve both good classification accuracy and precise latency constraint. For the operation-level, we present a bi-sampling search algorithm to moderate the operation collapse. For the depth-level, we introduce a sink-connecting search space to ensure the mutual exclusion between skip and other candidate operations, as well as eliminate the architecture redundancy. For the width-level, we propose an elasticity-scaling strategy that achieves precise latency constraint in a progressively fine-grained manner. Experiments on ImageNet demonstrate the effectiveness of TF-NAS. Particularly, our searched TF-NAS-A obtains 76.9% top-1 accuracy, achieving state-of-the-art results with less latency. The total search time is only 1.8 days on 1 Titan RTX GPU. Code is available at https://github.com/AberHu/TF-NAS.
