论文标题
反向丙胺：训练具有可区分神经调节性的自我修饰神经网络
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
论文作者
论文摘要
动物大脑中令人印象深刻的终身学习主要是由突触连通性的塑料变化促进的。重要的是，这些变化不是被动的，而是由神经调节积极控制的，这本身就在大脑的控制之下。由此产生的大脑的自我修饰能力在学习和适应中起着重要作用，并且是生物增强学习的主要基础。在这里，我们首次显示具有这种神经调节性可塑性的人工神经网络可以通过梯度下降进行训练。扩展了以前的Hebbian可塑性的工作，我们提出了可区分可塑性的可区分配方。我们表明，神经编码的可塑性可以提高神经网络在强化学习和监督学习任务方面的性能。在一项任务中，在基准语言建模任务（控制参数数）上，具有数百万参数的神经编码塑料LSTM优于标准LSTM。我们得出的结论是，可塑性的可区分神经调节为训练神经网络提供了一个有力的新框架。
The impressive lifelong learning in animal brains is primarily enabled by plastic changes in synaptic connectivity. Importantly, these changes are not passive, but are actively controlled by neuromodulation, which is itself under the control of the brain. The resulting self-modifying abilities of the brain play an important role in learning and adaptation, and are a major basis for biological reinforcement learning. Here we show for the first time that artificial neural networks with such neuromodulated plasticity can be trained with gradient descent. Extending previous work on differentiable Hebbian plasticity, we propose a differentiable formulation for the neuromodulation of plasticity. We show that neuromodulated plasticity improves the performance of neural networks on both reinforcement learning and supervised learning tasks. In one task, neuromodulated plastic LSTMs with millions of parameters outperform standard LSTMs on a benchmark language modeling task (controlling for the number of parameters). We conclude that differentiable neuromodulation of plasticity offers a powerful new framework for training neural networks.
