论文标题
矩阵传感中隐式偏差的局限性：初始化等级事项
Limitations of Implicit Bias in Matrix Sensing: Initialization Rank Matters
论文作者
论文摘要
在矩阵传感中，我们首先从数值上识别对初始化等级的敏感性是梯度流的隐式偏差的新限制。我们将通过数学上部分量化这一现象，我们确定经验风险的梯度流对低级别的结果隐含偏见，并成功地学习了种植的低级数基质，前提是初始化是低秩和在特定的“捕获邻里”中。在局部改进结果中，此捕获邻居远大于相应的社区。前者包含所有具有零训练误差的模型，而后者是模型的一个小社区，其测试误差为零。这些新的见解使我们能够设计一种用于矩阵传感的替代算法，该算法补充了现有文献中主要主导的高级和接近零初始化方案。
In matrix sensing, we first numerically identify the sensitivity to the initialization rank as a new limitation of the implicit bias of gradient flow. We will partially quantify this phenomenon mathematically, where we establish that the gradient flow of the empirical risk is implicitly biased towards low-rank outcomes and successfully learns the planted low-rank matrix, provided that the initialization is low-rank and within a specific "capture neighborhood". This capture neighborhood is far larger than the corresponding neighborhood in local refinement results; the former contains all models with zero training error whereas the latter is a small neighborhood of a model with zero test error. These new insights enable us to design an alternative algorithm for matrix sensing that complements the high-rank and near-zero initialization scheme which is predominant in the existing literature.
