论文标题
带有加权时间延迟反馈的门控复发性神经网络
Gated Recurrent Neural Networks with Weighted Time-Delay Feedback
论文作者
论文摘要
在本文中，我们通过引入具有加权时间延迟反馈机制的门控复发单元（GRU），提出了一种新颖的方法来建模顺序数据中的长期依赖性。我们所提出的模型，名为$τ$ -gru，是一个经常性单元的连续时间公式的离散版本，该版本由延迟微分方程（DDES）控制。我们证明了连续时间模型的解决方案的存在和独特性，并表明所提出的反馈机制可以显着改善长期依赖性的建模。我们的经验结果表明，$τ$ -Gru在一系列任务上胜过最先进的复发单元和封闭式的复发体系结构，从而实现了更快的融合和更好的概括。
In this paper, we present a novel approach to modeling long-term dependencies in sequential data by introducing a gated recurrent unit (GRU) with a weighted time-delay feedback mechanism. Our proposed model, named $τ$-GRU, is a discretized version of a continuous-time formulation of a recurrent unit, where the dynamics are governed by delay differential equations (DDEs). We prove the existence and uniqueness of solutions for the continuous-time model and show that the proposed feedback mechanism can significantly improve the modeling of long-term dependencies. Our empirical results indicate that $τ$-GRU outperforms state-of-the-art recurrent units and gated recurrent architectures on a range of tasks, achieving faster convergence and better generalization.
