论文标题
通过丰富的观测值提取具有线性动力学的潜在状态表示
Extracting Latent State Representations with Linear Dynamics from Rich Observations
论文作者
论文摘要
最近，在简单的线性动态情况下，尤其是在线性二次调节剂等问题中，许多强化学习技术被证明可以证明可以保证。但是，实际上，许多强化学习问题试图直接从诸如图像之类的富度，高维表示的情况下学习政策。即使存在在正确的潜在表示（例如位置和速度）中线性线性的潜在动力学，富的表示可能是非线性的，并且可能包含无关的特征。在这项工作中，我们研究了一个模型，其中有一个隐藏的线性子空间，其中动力学是线性的。对于这样的模型，我们给出了一种有效的算法，用于用线性动力学提取线性子空间。然后，我们将思想扩展到提取非线性映射，并在简单的环境中以丰富的观察结果验证我们的方法的有效性。
Recently, many reinforcement learning techniques were shown to have provable guarantees in the simple case of linear dynamics, especially in problems like linear quadratic regulators. However, in practice, many reinforcement learning problems try to learn a policy directly from rich, high dimensional representations such as images. Even if there is an underlying dynamics that is linear in the correct latent representations (such as position and velocity), the rich representation is likely to be nonlinear and can contain irrelevant features. In this work we study a model where there is a hidden linear subspace in which the dynamics is linear. For such a model we give an efficient algorithm for extracting the linear subspace with linear dynamics. We then extend our idea to extracting a nonlinear mapping, and empirically verify the effectiveness of our approach in simple settings with rich observations.
