论文标题
不仅仅是准确性：朝着对象识别的信任机器学习接口
More Than Accuracy: Towards Trustworthy Machine Learning Interfaces for Object Recognition
论文作者
论文摘要
本文研究了识别图像中对象的机器学习（ML）系统可视化的用户体验。这很重要，因为即使是良好的系统也会以意外的方式失败，因为照片共享网站上的错误分类显示。在我们的研究中，我们将ML背景的用户暴露于三个具有不同准确性水平的系统的可视化。在访谈中，我们探讨了可视化如何帮助用户评估使用系统的准确性以及系统的可视化和准确性如何影响信任和依赖。我们发现，参与者不仅在评估ML系统时专注于准确性。他们还考虑了错误分类的可感知性和严重性，并且更喜欢看到预测的概率。语义上合理的错误被认为比难以置信的错误不那么严重，这意味着可以通过错误的类型传达系统的准确性。
This paper investigates the user experience of visualizations of a machine learning (ML) system that recognizes objects in images. This is important since even good systems can fail in unexpected ways as misclassifications on photo-sharing websites showed. In our study, we exposed users with a background in ML to three visualizations of three systems with different levels of accuracy. In interviews, we explored how the visualization helped users assess the accuracy of systems in use and how the visualization and the accuracy of the system affected trust and reliance. We found that participants do not only focus on accuracy when assessing ML systems. They also take the perceived plausibility and severity of misclassification into account and prefer seeing the probability of predictions. Semantically plausible errors are judged as less severe than errors that are implausible, which means that system accuracy could be communicated through the types of errors.
