论文标题
基于粒子的变异推断，具有预处理的功能梯度流
Particle-based Variational Inference with Preconditioned Functional Gradient Flow
论文作者
论文摘要
基于粒子的变异推理（VI）可以最大程度地减少模型样本与目标后验之间的KL差异，并具有梯度流量估计。随着Stein变分梯度下降（SVGD）的普及，基于粒子的VI算法的焦点一直在重现核Hilbert Space（RKHS）中的功能属性上，以近似梯度流。但是，RKHS的需求限制了功能类和算法灵活性。本文通过引入一个涵盖RKHS Norm作为特殊情况的功能正则化术语来为此问题提供了一个通用的解决方案。这使我们能够提出一种新的基于粒子的VI算法，称为预处理功能梯度流（PFG）。与SVGD相比，PFG具有多个优点。它具有较大的功能类别，在大粒径场景中提高了可伸缩性，更好地适应了条件不良的分布，以及可证明的KL差异中可证明的连续时间收敛。另外，可以合并非线性功能类别（例如神经网络）以估计梯度流。我们的理论和实验证明了所提出的框架的有效性。
Particle-based variational inference (VI) minimizes the KL divergence between model samples and the target posterior with gradient flow estimates. With the popularity of Stein variational gradient descent (SVGD), the focus of particle-based VI algorithms has been on the properties of functions in Reproducing Kernel Hilbert Space (RKHS) to approximate the gradient flow. However, the requirement of RKHS restricts the function class and algorithmic flexibility. This paper offers a general solution to this problem by introducing a functional regularization term that encompasses the RKHS norm as a special case. This allows us to propose a new particle-based VI algorithm called preconditioned functional gradient flow (PFG). Compared to SVGD, PFG has several advantages. It has a larger function class, improved scalability in large particle-size scenarios, better adaptation to ill-conditioned distributions, and provable continuous-time convergence in KL divergence. Additionally, non-linear function classes such as neural networks can be incorporated to estimate the gradient flow. Our theory and experiments demonstrate the effectiveness of the proposed framework.
