论文标题
利用关键信息建模来通过二元性微调改善较小的DATA限制性新闻标题生成
Leveraging Key Information Modeling to Improve Less-Data Constrained News Headline Generation via Duality Fine-Tuning
论文作者
论文摘要
最近的语言生成模型主要是在大规模数据集上培训的，而在某些实际情况下，培训数据集通常很昂贵，并且将是小规模的。在本文中，我们调查了少于DATA限制的发电的挑战性任务，尤其是当读者预期生成的新闻头条简短但期望读者同时保持可读性和信息性时。我们通过正式定义关键信息预测和标题生成任务之间的概率二元性约束来强调关键信息建模任务，并提出一种新颖的双重性微调方法。所提出的方法可以从有限的数据中捕获更多信息，在单独的任务之间建立连接，并且适合DATA限制的生成任务。此外，该方法可以利用各种预训练的生成方案，例如自动回归和编码器模型。我们进行了广泛的实验，以证明我们的方法有效而有效，可以在两个公共数据集上的语言建模和信息性正确性度量方面提高性能。
Recent language generative models are mostly trained on large-scale datasets, while in some real scenarios, the training datasets are often expensive to obtain and would be small-scale. In this paper we investigate the challenging task of less-data constrained generation, especially when the generated news headlines are short yet expected by readers to keep readable and informative simultaneously. We highlight the key information modeling task and propose a novel duality fine-tuning method by formally defining the probabilistic duality constraints between key information prediction and headline generation tasks. The proposed method can capture more information from limited data, build connections between separate tasks, and is suitable for less-data constrained generation tasks. Furthermore, the method can leverage various pre-trained generative regimes, e.g., autoregressive and encoder-decoder models. We conduct extensive experiments to demonstrate that our method is effective and efficient to achieve improved performance in terms of language modeling metric and informativeness correctness metric on two public datasets.
