论文标题
在空间上探索概念贡献：使用空间激活概念向量的隐藏层解释
Exploring Concept Contribution Spatially: Hidden Layer Interpretation with Spatial Activation Concept Vector
论文作者
论文摘要
要解释深度学习模型，一个主流是通过网络探索学习的概念。使用概念激活向量（TCAV）进行测试提供了一种强大的工具，可以量化查询概念（用用户定义的指导图像表示）对目标类别的贡献。例如，我们可以定量评估概念条纹是否有助于使用TCAV模型的预测斑马。因此，TCAV使深网的推理过程变得更白。它已应用于解决诸如诊断之类的实际问题。但是，对于某些图像，目标对象仅占据该区域的一小部分，TCAV评估可能会受到冗余背景特征的干预，因为TCAV计算基于整个隐藏层的概念对目标类的贡献。为了解决此问题，基于TCAV，我们提出了空间激活概念向量（SACV），该空间激活概念向量（SACV）在评估其对目标类别模型预测的贡献的同时，将相关的空间位置识别为查询概念。实验表明，SACV为隐藏层生成了更细粒度的解释图，并在空间上量化了概念的贡献。此外，它避免了对背景功能的干扰。该代码可在https://github.com/antonotnawang/spatial-activation-concept-vector上获得。
To interpret deep learning models, one mainstream is to explore the learned concepts by networks. Testing with Concept Activation Vector (TCAV) presents a powerful tool to quantify the contribution of query concepts (represented by user-defined guidance images) to a target class. For example, we can quantitatively evaluate whether and to what extent concept striped contributes to model prediction zebra with TCAV. Therefore, TCAV whitens the reasoning process of deep networks. And it has been applied to solve practical problems such as diagnosis. However, for some images where the target object only occupies a small fraction of the region, TCAV evaluation may be interfered with by redundant background features because TCAV calculates concept contribution to a target class based on a whole hidden layer. To tackle this problem, based on TCAV, we propose Spatial Activation Concept Vector (SACV) which identifies the relevant spatial locations to the query concept while evaluating their contributions to the model prediction of the target class. Experiment shows that SACV generates a more fine-grained explanation map for a hidden layer and quantifies concepts' contributions spatially. Moreover, it avoids interference from background features. The code is available on https://github.com/AntonotnaWang/Spatial-Activation-Concept-Vector.
