论文标题
重新访问过程与产品指标：大规模分析
Revisiting Process versus Product Metrics: a Large Scale Analysis
论文作者
论文摘要
许多方法可以从软件数据中构建预测模型。但是，随着我们从分析中的小小的（处理少数项目）转变为一定程度的分析（处理数百个项目）时，我们应该认可哪些方法和结论？ 为了回答这个问题，我们使用700个GitHub项目中的722,471名提交了先前的小规模结果（关于缺陷预测和指标粒度的过程与产品指标）。我们发现，在扩大分析的一般性分析时，一些分析得出的结论仍然存在。例如，像先前的工作一样，我们看到该过程指标比产品指标更好地预测缺陷（最佳基于过程/基于产品的学习者分别实现了98 \％/44 \％的召回召回，而AUC的AUC和95 \％/54 \％的AUC，中位数为中位数）。 也就是说，我们警告说，信任指标重要性是从分析中的研究结果导致的，因为当这些研究转向一般性分析时，它们会发生巨大变化。同样，当推理在大约数百个项目中，最好使用来自多个模型的预测（因为单个模型预测会变得困惑并表现出很大的差异）。
Numerous methods can build predictive models from software data. However, what methods and conclusions should we endorse as we move from analytics in-the-small (dealing with a handful of projects) to analytics in-the-large (dealing with hundreds of projects)? To answer this question, we recheck prior small-scale results (about process versus product metrics for defect prediction and the granularity of metrics) using 722,471 commits from 700 Github projects. We find that some analytics in-the-small conclusions still hold when scaling up to analytics in-the-large. For example, like prior work, we see that process metrics are better predictors for defects than product metrics (best process/product-based learners respectively achieve recalls of 98\%/44\% and AUCs of 95\%/54\%, median values). That said, we warn that it is unwise to trust metric importance results from analytics in-the-small studies since those change dramatically when moving to analytics in-the-large. Also, when reasoning in-the-large about hundreds of projects, it is better to use predictions from multiple models (since single model predictions can become confused and exhibit a high variance).
