论文标题
贝叶斯A/B测试业务决策
Bayesian A/B Testing for Business Decisions
论文作者
论文摘要
对照实验（A/B测试或随机现场实验）是事实上的标准，可以在实施变化和观察客户响应时做出数据驱动的决策。分析此类实验的方法对于产品和营销经理等利益相关者来说应该很容易理解。贝叶斯推论最近获得了很多知名度，就A/B测试而言，一个关键论点是简单的解释性。对于利益相关者，“最好的概率”（具有相应的可靠间隔）提供了一个自然的指标来做出业务决策。在本文中，我们激发了企业主通常存在的典型问题以及如何通过贝叶斯方法回答它们。我们介绍了三个实验场景，这些场景在我们公司中很常见，如何以贝叶斯方式建模以及如何使用模型来制定业务决策。对于每种情况，我们都会提出一个真实的实验，结果和最终的业务决策。
Controlled experiments (A/B tests or randomized field experiments) are the de facto standard to make data-driven decisions when implementing changes and observing customer responses. The methodology to analyze such experiments should be easily understandable to stakeholders like product and marketing managers. Bayesian inference recently gained a lot of popularity and, in terms of A/B testing, one key argument is the easy interpretability. For stakeholders, "probability to be best" (with corresponding credible intervals) provides a natural metric to make business decisions. In this paper, we motivate the quintessential questions a business owner typically has and how to answer them with a Bayesian approach. We present three experiment scenarios that are common in our company, how they are modeled in a Bayesian fashion, and how to use the models to draw business decisions. For each of the scenarios, we present a real-world experiment, the results and the final business decisions drawn.
