论文标题
您的损失很可能是可能的
It Is Likely That Your Loss Should be a Likelihood
论文作者
论文摘要
许多常见的损失函数，例如均方纠正，跨凝结和重建损失，这是不必要的刚性。在概率解释下，这些常见损失对应于具有固定形状和尺度的分布。相反，我们主张优化包括正常方差和SoftMax温度等参数的完整可能性。与模型参数对这些“似然参数”的关节优化还可以适应损失的尺度和形状，除了正则化强度。我们探索并系统地评估如何参数化和应用似然参数以进行鲁棒建模，离群检测和重新校准。此外，我们建议通过安装正常和拉普拉斯先验的比例参数，并引入更灵活的元素正规化器，从而适应$ l_2 $和$ l_1 $重量。
Many common loss functions such as mean-squared-error, cross-entropy, and reconstruction loss are unnecessarily rigid. Under a probabilistic interpretation, these common losses correspond to distributions with fixed shapes and scales. We instead argue for optimizing full likelihoods that include parameters like the normal variance and softmax temperature. Joint optimization of these "likelihood parameters" with model parameters can adaptively tune the scales and shapes of losses in addition to the strength of regularization. We explore and systematically evaluate how to parameterize and apply likelihood parameters for robust modeling, outlier-detection, and re-calibration. Additionally, we propose adaptively tuning $L_2$ and $L_1$ weights by fitting the scale parameters of normal and Laplace priors and introduce more flexible element-wise regularizers.
