论文标题
学习排名学习曲线
Learning to Rank Learning Curves
论文作者
论文摘要
许多自动化的机器学习方法，例如用于高参数和神经体系结构优化的方法在计算上都是昂贵的，因为它们涉及培训许多不同的模型配置。在这项工作中，我们提出了一种新方法，该方法通过在培训的早期终止配置来节省计算预算。与现有方法相反，我们将此任务视为排名和转移学习问题。我们定性地表明，通过优化来自其他数据集的成对排名损失并利用学习曲线，我们的模型能够有效地对学习曲线进行排名，而无需观察许多或很长的学习曲线。我们进一步证明，我们的方法可用于加速神经体系结构搜索，而无需对发现的体系结构进行显着性能下降。在进一步的实验中，我们分析了排名的质量，不同模型组件的影响以及模型的预测行为。
Many automated machine learning methods, such as those for hyperparameter and neural architecture optimization, are computationally expensive because they involve training many different model configurations. In this work, we present a new method that saves computational budget by terminating poor configurations early on in the training. In contrast to existing methods, we consider this task as a ranking and transfer learning problem. We qualitatively show that by optimizing a pairwise ranking loss and leveraging learning curves from other datasets, our model is able to effectively rank learning curves without having to observe many or very long learning curves. We further demonstrate that our method can be used to accelerate a neural architecture search by a factor of up to 100 without a significant performance degradation of the discovered architecture. In further experiments we analyze the quality of ranking, the influence of different model components as well as the predictive behavior of the model.
