Power Analysis for Conjoint Experiments

Abstract

Conjoint experiments aiming to estimate average marginal component effects and related quantities have become a standard tool for social scientists. However, existing solutions for power analyses to find appropriate sample sizes for such studies have various shortcomings and accordingly, explicit sample size planning is rare. Based on recent advances in statistical inference for factorial experiments, we derive simple yet generally applicable formulae to calculate power and minimum required sample sizes for testing average marginal component effects (AMCEs), conditional AMCEs, as well as interaction effects in forced-choice conjoint experiments. The only input needed are expected effect sizes. Our approach only assumes random sampling of individuals or randomization of profiles and avoids any parametric assumption. Furthermore, we show that clustering standard errors on individuals is not necessary and does not affect power. Our results caution against designing conjoint experiments with small sample sizes, especially for detecting heterogeneity and interactions. We provide an R package that implements our approach.