论文标题
利用来自变形金刚的双向编码器表示来选择
Utilizing Bidirectional Encoder Representations from Transformers for Answer Selection
论文作者
论文摘要
近年来，在大型数据集中预先培训基于变压器的模型，用于大型数据集中的语言建模任务，然后对下游任务进行微调。这种预训练的语言模型的一个主要优点是，它们可以有效地吸收句子中每个单词的上下文。但是，对于诸如答案选择任务之类的任务，尚未广泛使用预训练的语言模型。为了调查它们在此类任务中的有效性，在本文中，我们采用了来自Transformer（BERT）语言模型的预先训练的双向编码器表示，并将其用于两个问题答录（QA）数据集（QA）数据集和三个社区问题答录（CQA）数据集（CQA）数据集（CQA）数据集来选择答案选择任务。我们发现，针对答案选择任务的BERT模型非常有效，并且与以前的最新时间相比，QA数据集中的最大提高了13.1％，CQA数据集的最大提高为13.1％。
Pre-training a transformer-based model for the language modeling task in a large dataset and then fine-tuning it for downstream tasks has been found very useful in recent years. One major advantage of such pre-trained language models is that they can effectively absorb the context of each word in a sentence. However, for tasks such as the answer selection task, the pre-trained language models have not been extensively used yet. To investigate their effectiveness in such tasks, in this paper, we adopt the pre-trained Bidirectional Encoder Representations from Transformer (BERT) language model and fine-tune it on two Question Answering (QA) datasets and three Community Question Answering (CQA) datasets for the answer selection task. We find that fine-tuning the BERT model for the answer selection task is very effective and observe a maximum improvement of 13.1% in the QA datasets and 18.7% in the CQA datasets compared to the previous state-of-the-art.
