论文标题
光流蒸馏：迈向高效稳定的视频风格转移
Optical Flow Distillation: Towards Efficient and Stable Video Style Transfer
论文作者
论文摘要
视频风格转移技术激发了移动设备上的许多令人兴奋的应用程序。但是，它们的效率和稳定性远非令人满意。为了提高跨帧的传输稳定性，尽管具有较高的计算复杂性，例如占用超过97％的推理时间。本文提议通过知识蒸馏范式学习轻巧的视频传输网络。我们采用了两个教师网络，其中一个在推理期间采用光流，而另一个则没有。这两个教师网络之间的输出差异突出了光流进行的改进，然后采用了这些方法来提炼目标学生网络。此外，通过模仿输入视频的等级，采用低级蒸馏损失来稳定学生网络的产出。广泛的实验表明，没有光流模块的学生网络仍然能够生成稳定的视频，并且运行速度比教师网络快得多。
Video style transfer techniques inspire many exciting applications on mobile devices. However, their efficiency and stability are still far from satisfactory. To boost the transfer stability across frames, optical flow is widely adopted, despite its high computational complexity, e.g. occupying over 97% inference time. This paper proposes to learn a lightweight video style transfer network via knowledge distillation paradigm. We adopt two teacher networks, one of which takes optical flow during inference while the other does not. The output difference between these two teacher networks highlights the improvements made by optical flow, which is then adopted to distill the target student network. Furthermore, a low-rank distillation loss is employed to stabilize the output of student network by mimicking the rank of input videos. Extensive experiments demonstrate that our student network without an optical flow module is still able to generate stable video and runs much faster than the teacher network.
