论文标题
RNN和LSTM的记忆力很长吗？
Do RNN and LSTM have Long Memory?
论文作者
论文摘要
提议LSTM网络克服学习长期依赖的困难，并在应用程序上取得了重大进步。考虑到它的成功和缺点，本文提出了一个问题 - RNN和LSTM是否有很长的记忆？我们通过证明RNN和LSTM从统计角度没有长期记忆来部分回答。进一步引入了长期内存网络的新定义，它要求模型权重以多项式速率衰减。为了验证我们的理论，我们通过最少的修改将RNN和LSTM转换为长存储网络，并在建模各种数据集的长期依赖性时说明了它们的优势。
The LSTM network was proposed to overcome the difficulty in learning long-term dependence, and has made significant advancements in applications. With its success and drawbacks in mind, this paper raises the question - do RNN and LSTM have long memory? We answer it partially by proving that RNN and LSTM do not have long memory from a statistical perspective. A new definition for long memory networks is further introduced, and it requires the model weights to decay at a polynomial rate. To verify our theory, we convert RNN and LSTM into long memory networks by making a minimal modification, and their superiority is illustrated in modeling long-term dependence of various datasets.
