论文标题
使用交叉任务最近的邻居使用数据有效的鉴定
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors
论文作者
论文摘要
获取标记的数据以训练模型以完成感兴趣的任务通常很昂贵。先前的工作显示了多任务数据增强的培训模型，并有效地将知识转移到新任务。为了有效地构建特定于任务的模型，我们假设访问少量的未标记目标任务示例（32-1000）示例，并使用这些示例从大量的多任务数据池中检索最相似的标记示例，并带有提示。与当前在均匀采样的固定模型的实践相比，促使多任务数据（例如：Flan，t0），我们在最近的交叉任务中的鉴定方法显着提高了数据效率。我们的模型仅使用来自P3池的2％数据，而没有任何标记的目标任务数据，在14个数据集中，有12个数据集中的12个数据集中，我们的模型优于对所有可用数据培训的强大基准，这些基线代表了包括法律和科学文档QA在内的持有任务。同样，在超自然结构的最近邻居中训练的模型，约占池的5％，从该池中的12个持有任务上获得了可比的性能。此外，与我们的方法相比，我们的方法生产的模型还提供了更好的初始化，该模型比单个多任务框架模型在目标任务数据上进行了少量填充，如8个数据集中的几个射击列出的T0-3B模型所示，相对改进为2-23％。
Obtaining labeled data to train a model for a task of interest is often expensive. Prior work shows training models on multitask data augmented with task descriptions (prompts) effectively transfers knowledge to new tasks. Towards efficiently building task-specific models, we assume access to a small number (32-1000) of unlabeled target-task examples and use those to retrieve the most similar labeled examples from a large pool of multitask data augmented with prompts. Compared to the current practice of finetuning models on uniformly sampled prompted multitask data (e.g.: FLAN, T0), our approach of finetuning on cross-task nearest neighbors is significantly more data-efficient. Using only 2% of the data from the P3 pool without any labeled target-task data, our models outperform strong baselines trained on all available data by 3-30% on 12 out of 14 datasets representing held-out tasks including legal and scientific document QA. Similarly, models trained on cross-task nearest neighbors from SuperNaturalInstructions, representing about 5% of the pool, obtain comparable performance to state-of-the-art models on 12 held-out tasks from that pool. Moreover, the models produced by our approach also provide a better initialization than single multitask finetuned models for few-shot finetuning on target-task data, as shown by a 2-23% relative improvement over few-shot finetuned T0-3B models on 8 datasets.
