论文标题
奖励对离线多代理强化学习的中毒攻击
Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
论文作者
论文摘要
在离线多代理增强学习（MARL）中，代理商估算给定数据集的政策。我们在这种情况下研究了奖励供电攻击，在这种情况下，外源攻击者会在代理商看到数据集之前修改数据集中的奖励。攻击者希望将每个代理商带入邪恶的目标政策，同时最大程度地减少奖励修改的$ l^p $规范。与攻击单代理RL不同，我们表明攻击者可以将目标策略安装为马尔可夫完美的统治策略平衡（MPDSE），保证理性的代理人可以遵循。这种攻击可以比单独的单一代理攻击便宜得多。我们表明，攻击对包括不确定性的学习者在内的各种MARL代理作用，并且我们展示了线性程序以有效解决攻击问题。我们还研究了数据集结构与最低攻击成本之间的关系。我们的工作为在离线MAL中学习防御铺平了道路。
In offline multi-agent reinforcement learning (MARL), agents estimate policies from a given dataset. We study reward-poisoning attacks in this setting where an exogenous attacker modifies the rewards in the dataset before the agents see the dataset. The attacker wants to guide each agent into a nefarious target policy while minimizing the $L^p$ norm of the reward modification. Unlike attacks on single-agent RL, we show that the attacker can install the target policy as a Markov Perfect Dominant Strategy Equilibrium (MPDSE), which rational agents are guaranteed to follow. This attack can be significantly cheaper than separate single-agent attacks. We show that the attack works on various MARL agents including uncertainty-aware learners, and we exhibit linear programs to efficiently solve the attack problem. We also study the relationship between the structure of the datasets and the minimal attack cost. Our work paves the way for studying defense in offline MARL.
