论文标题
crepe：开放域问题，用虚假的预设回答
CREPE: Open-Domain Question Answering with False Presuppositions
论文作者
论文摘要
寻求用户的信息通常会以虚假的预设提出问题，尤其是在询问陌生主题时。 Most existing question answering (QA) datasets, in contrast, assume all questions have well defined answers.我们介绍Crepe，这是一个质量检查数据集，其中包含在线信息寻求论坛中的预设失败的自然分布。我们发现25％的问题包含虚假的预设，并为这些预设及其更正提供注释。通过广泛的基线实验，我们表明，现有的开放域质量主模型的适应能力可以很好地找到前提，但是在预测预设是否正确地正确的情况下进行了挣扎。这在很大程度上是由于难以从大型文本语料库中检索相关的证据段落。 Crepe提供了一个基准来研究野外回答的问题，我们的分析为未来的工作提供了更好的建模和进一步研究任务的途径。
Information seeking users often pose questions with false presuppositions, especially when asking about unfamiliar topics. Most existing question answering (QA) datasets, in contrast, assume all questions have well defined answers. We introduce CREPE, a QA dataset containing a natural distribution of presupposition failures from online information-seeking forums. We find that 25% of questions contain false presuppositions, and provide annotations for these presuppositions and their corrections. Through extensive baseline experiments, we show that adaptations of existing open-domain QA models can find presuppositions moderately well, but struggle when predicting whether a presupposition is factually correct. This is in large part due to difficulty in retrieving relevant evidence passages from a large text corpus. CREPE provides a benchmark to study question answering in the wild, and our analyses provide avenues for future work in better modeling and further studying the task.
