论文标题
使用高阶类别理论统一因果推理和加强学习
Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theory
论文作者
论文摘要
我们提出了一种统一的形式主义，用于使用高阶类别理论的结构发现因果模型和预测状态表示（PSR）模型（RL）。具体而言，我们使用Simplicial对象将序数类别类别的符号函数（违反函数）模拟在两个设置中的结构发现。在条件独立性下等效的因果模型的片段（定义为因果角）以及预测状态表示中潜在测试的子序列（定义为预测角）都是由内部和面对面相对的特定顶部的emoval造成的简单物体的特殊情况，是一个简单物体的特殊情况。两种设置中的潜在结构发现都涉及相同的基本数学问题，即通过解决通勤图中的提升问题，并利用定义高阶对称性的弱同质性来查找简单对象的角的扩展。解决“内部”与“外部”喇叭问题的解决方案导致了高阶类别的各种概念，包括弱KAN复合物和准游戏。我们根据通用因果模型或通用决策模型及其简单对象表示的类别之间的伴随函数来定义两个设置中结构发现的抽象问题。
We present a unified formalism for structure discovery of causal models and predictive state representation (PSR) models in reinforcement learning (RL) using higher-order category theory. Specifically, we model structure discovery in both settings using simplicial objects, contravariant functors from the category of ordinal numbers into any category. Fragments of causal models that are equivalent under conditional independence -- defined as causal horns -- as well as subsequences of potential tests in a predictive state representation -- defined as predictive horns -- are both special cases of horns of a simplicial object, subsets resulting from the removal of the interior and the face opposite a particular vertex. Latent structure discovery in both settings involve the same fundamental mathematical problem of finding extensions of horns of simplicial objects through solving lifting problems in commutative diagrams, and exploiting weak homotopies that define higher-order symmetries. Solutions to the problem of filling "inner" vs "outer" horns leads to various notions of higher-order categories, including weak Kan complexes and quasicategories. We define the abstract problem of structure discovery in both settings in terms of adjoint functors between the category of universal causal models or universal decision models and its simplicial object representation.
