论文标题
大型语言模型的语义无标记启动
Semantic-Oriented Unlabeled Priming for Large-Scale Language Models
论文作者
论文摘要
由于与大型语言模型相关的高成本，各种最近的作品建议将它们适应特定任务，而无需通过文本学习任何参数更新。不幸的是，对于中文学习，目前无法利用未标记的数据，这通常比标记的示例容易得多。因此，在这项工作中，我们调查了使用未标记的示例来改善未经任何填补的零声学模型的零拍摄性能：我们介绍了以语义为导向的无标记的启动（汤），这种方法通过检索语义上类似的未标记示例来对示例进行分类，以将标签分配给它们，然后将其分配给它们，然后将其分配给它们，然后将其分配给它们，然后将其分配给它们，然后将其分配给它们。我们还提出了更适合我们的设置的新启动策略，这是一种新的启动策略，可以使用更多的示例，而不是适合上下文窗口。
Due to the high costs associated with finetuning large language models, various recent works propose to adapt them to specific tasks without any parameter updates through in-context learning. Unfortunately, for in-context learning there is currently no way to leverage unlabeled data, which is often much easier to obtain in large quantities than labeled examples. In this work, we therefore investigate ways to make use of unlabeled examples to improve the zero-shot performance of pretrained language models without any finetuning: We introduce Semantic-Oriented Unlabeled Priming (SOUP), a method that classifies examples by retrieving semantically similar unlabeled examples, assigning labels to them in a zero-shot fashion, and then using them for in-context learning. We also propose bag-of-contexts priming, a new priming strategy that is more suitable for our setting and enables the usage of more examples than fit into the context window.
