论文标题
在指定实体识别中应用预训练模型
Application of Pre-training Models in Named Entity Recognition
论文作者
论文摘要
命名实体识别（NER）是一项基本的自然语言处理（NLP）任务，可从非结构化数据中提取实体。以前的NER方法基于机器学习或深度学习。最近，培训模型在多个NLP任务上的性能显着提高。在本文中，首先，我们介绍了四种常见的预训练模型的建筑和预训练任务：Bert，Ernie，Ernie2.0-tiny和Roberta。然后，我们通过微调将这些预训练模型应用于NER任务，并比较不同模型体系结构和预训练任务对NER任务的影响。实验结果表明，罗伯塔（Roberta）在MSRA-2006数据集上取得了最新的结果。
Named Entity Recognition (NER) is a fundamental Natural Language Processing (NLP) task to extract entities from unstructured data. The previous methods for NER were based on machine learning or deep learning. Recently, pre-training models have significantly improved performance on multiple NLP tasks. In this paper, firstly, we introduce the architecture and pre-training tasks of four common pre-training models: BERT, ERNIE, ERNIE2.0-tiny, and RoBERTa. Then, we apply these pre-training models to a NER task by fine-tuning, and compare the effects of the different model architecture and pre-training tasks on the NER task. The experiment results showed that RoBERTa achieved state-of-the-art results on the MSRA-2006 dataset.
