论文标题
一个简单但有效的可插入实体查找表，用于预训练的语言模型
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
论文作者
论文摘要
预训练的语言模型（PLM）不能很好地回忆起大规模语料库中展示的实体的丰富事实知识，尤其是那些稀有实体。在本文中，我们建议通过汇总该语料库中多个事件的实体输出表示形式来按需建立一个简单但有效的可插入实体查找表（PELT）。可以将PELT兼容，作为输入中补充实体知识的输入。与以前的知识增强的PLM相比，PELT仅需要0.2％-5％的预发行，并且能够从域外传统中获取知识以进行域适应方案。与知识相关的任务的实验表明，我们的方法（PELT）可以灵活有效地将实体知识从相关的Corpora转移到具有不同架构的PLM中。
Pre-trained language models (PLMs) cannot well recall rich factual knowledge of entities exhibited in large-scale corpora, especially those rare entities. In this paper, we propose to build a simple but effective Pluggable Entity Lookup Table (PELT) on demand by aggregating the entity's output representations of multiple occurrences in the corpora. PELT can be compatibly plugged as inputs to infuse supplemental entity knowledge into PLMs. Compared to previous knowledge-enhanced PLMs, PELT only requires 0.2%-5% pre-computation with capability of acquiring knowledge from out-of-domain corpora for domain adaptation scenario. The experiments on knowledge-related tasks demonstrate that our method, PELT, can flexibly and effectively transfer entity knowledge from related corpora into PLMs with different architectures.
