论文标题
通过双层优化的光流和自我的无监督学习
Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization
论文作者
论文摘要
我们通过将几何约束纳入无监督的深度学习框架中，解决了刚性场景中关节光流和相机运动估计的问题。与依靠亮度恒定的现有方法和局部平滑度进行光流估计不同，我们利用光流和相机运动之间使用表现几何形状来利用全球关系。特别是，我们将光流和摄像机运动的预测作为双层优化问题，由高级问题组成，以估算符合预测的摄像机运动的流量，以及鉴于预测的光流，估算摄像机运动的较低级别问题。我们使用隐式差异化来通过低级几何优化层与其实现无关，从而使网络的端到端训练可以进行反向传播。借助全球执行的几何约束，我们能够在具有挑战性的情况下提高估计的光流的质量，并获得与其他无监督学习方法相比，获得更好的相机运动估计。
We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brightness constancy and local smoothness for optical flow estimation, we exploit the global relationship between optical flow and camera motion using epipolar geometry. In particular, we formulate the prediction of optical flow and camera motion as a bi-level optimization problem, consisting of an upper-level problem to estimate the flow that conforms to the predicted camera motion, and a lower-level problem to estimate the camera motion given the predicted optical flow. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation, allowing end-to-end training of the network. With globally-enforced geometric constraints, we are able to improve the quality of the estimated optical flow in challenging scenarios and obtain better camera motion estimates compared to other unsupervised learning methods.
