Beyond reweighting: On the predictive role of covariate shift in effect generalization

Generalizing scientific findings across populations is central to science and policy. A common approach is to reweight observed covariates, implicitly assuming that only the covariate distribution changes across settings. Analyzing 680 studies across 65 sites, we show that shifts in the outcome conditional on covariates (Y | X) are common, but that the strength of these shifts can be predicted from changes in observed covariates. I'll introduce a statistical framework that explains this predictive, rather than merely explanatory, role of covariates in effect generalization, and discuss how it can guide the development of new methods for external validity.

This is joint work with Ying Jin, Naoki Egami, Yujin Jeong, Anna Lyubarskaja, and Ivy Zhang.