论文标题
基于混合模型的强化学习用于细胞疗法制造过程控制的机会
Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control
论文作者
论文摘要
在细胞疗法制造的主要挑战（包括高复杂性，高不确定性和非常有限的过程观测值）的驱动下，我们提出了一种基于混合模型的增强学习（RL）来有效指导过程控制。我们首先创建了一个概率知识图（kg）混合模型，该模型表征了基于风险和科学的生物制造过程机制的理解并量化固有的随机性，例如批处理变化。它可以捕获关键特征，包括非线性反应，非平稳动力学和部分观察到的状态。该混合模型可以利用现有的机械模型并促进从异质过程数据中学习。计算采样方法用于生成量化模型不确定性的后样品。然后，我们介绍了基于混合模型的贝叶斯RL，既考虑固有的随机性和模型不确定性，又可以指导最佳，健壮和可解释的动态决策。细胞疗法制造实例用于从经验上证明，所提出的框架可以优于经典的确定性机械模型辅助过程优化。
Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently guide process control. We first create a probabilistic knowledge graph (KG) hybrid model characterizing the risk- and science-based understanding of biomanufacturing process mechanisms and quantifying inherent stochasticity, e.g., batch-to-batch variation. It can capture the key features, including nonlinear reactions, nonstationary dynamics, and partially observed state. This hybrid model can leverage existing mechanistic models and facilitate learning from heterogeneous process data. A computational sampling approach is used to generate posterior samples quantifying model uncertainty. Then, we introduce hybrid model-based Bayesian RL, accounting for both inherent stochasticity and model uncertainty, to guide optimal, robust, and interpretable dynamic decision making. Cell therapy manufacturing examples are used to empirically demonstrate that the proposed framework can outperform the classical deterministic mechanistic model assisted process optimization.
