论文标题
是语言不可思议的意思是什么？探测类型学特性的多语言句子编码器
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties
论文作者
论文摘要
多语言句子编码器在下游NLP任务中的跨语性模型转移中已经取得了很大的成功。但是，我们对单个语言的属性或它们所编码的语言变化的一般模式的了解相对较少。我们提出了从最先进的多语言编码器（激光，M-Bert，XLM和XLM-R）探测句子表示的方法，相对于与词汇，形态和句法结构有关的一系列类型学特性。此外，我们研究了如何将这些信息分布在模型的所有层中。我们的结果表明，在编码与不同训练策略相关的语言差异方面存在有趣的差异。
Multilingual sentence encoders have seen much success in cross-lingual model transfer for downstream NLP tasks. Yet, we know relatively little about the properties of individual languages or the general patterns of linguistic variation that they encode. We propose methods for probing sentence representations from state-of-the-art multilingual encoders (LASER, M-BERT, XLM and XLM-R) with respect to a range of typological properties pertaining to lexical, morphological and syntactic structure. In addition, we investigate how this information is distributed across all layers of the models. Our results show interesting differences in encoding linguistic variation associated with different pretraining strategies.
