论文标题
非线性最小二乘问题的可扩展无导数优化
Scalable Derivative-Free Optimization for Nonlinear Least-Squares Problems
论文作者
论文摘要
无衍生物 - 或零订单优化（DFO）已因其在包括机器学习在内的各种应用领域解决问题的能力而引起了人们的关注，尤其是涉及随机和/或计算昂贵的目标。在这项工作中，我们开发了一种基于模型的新型DFO方法来解决非线性最小二乘问题。我们通过使用草图方法在观察空间中降低维度，从而避免构建完整的本地模型，从而改善了最先进的DFO。我们的方法具有触电计算成本，在大数据制度中在问题维度上是线性的，数值证据表明，与现有软件相比，它在过度确定的最小二乘问题上已经大大提高了运行时性能。
Derivative-free - or zeroth-order - optimization (DFO) has gained recent attention for its ability to solve problems in a variety of application areas, including machine learning, particularly involving objectives which are stochastic and/or expensive to compute. In this work, we develop a novel model-based DFO method for solving nonlinear least-squares problems. We improve on state-of-the-art DFO by performing dimensionality reduction in the observational space using sketching methods, avoiding the construction of a full local model. Our approach has a per-iteration computational cost which is linear in problem dimension in a big data regime, and numerical evidence demonstrates that, compared to existing software, it has dramatically improved runtime performance on overdetermined least-squares problems.
