论文标题
PERL：基于枢轴的域适应预先训练的深层上下文化嵌入模型
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
论文作者
论文摘要
基于枢轴的神经表示模型已导致NLP的域适应性取得重大进展。但是，遵循这种方法的先前工作仅利用来自源域和来自源域和目标域的未标记数据的标记数据，但忽略了不一定是从这些域中绘制的大量未标记的Corpora。为了减轻这一点，我们提出了PERL：一种代表学习模型，该模型扩展了上下文化的单词嵌入模型，例如基于枢轴的微调BERT。 Perl在22个情感分类域适应设置中优于强大的基线，改善了内域模型性能，产生有效的减小尺寸模型并提高模型稳定性。
Pivot-based neural representation models have lead to significant progress in domain adaptation for NLP. However, previous works that follow this approach utilize only labeled data from the source domain and unlabeled data from the source and target domains, but neglect to incorporate massive unlabeled corpora that are not necessarily drawn from these domains. To alleviate this, we propose PERL: A representation learning model that extends contextualized word embedding models such as BERT with pivot-based fine-tuning. PERL outperforms strong baselines across 22 sentiment classification domain adaptation setups, improves in-domain model performance, yields effective reduced-size models and increases model stability.
