Relationships Between Shared Group Properties: Theory, Measurement, Estimation, and Adjustment

Abstract

Teams researchers, meta‐analysts, and others often study relationships between group‐level constructs measured by aggregating individual‐level variables, but it is well‐known that group‐mean correlations are influenced by individual‐level relationships. We review multilevel issues and current practices in estimating and adjusting group‐mean associations. We give special attention to comparisons across levels of analysis, and to adjustments for estimating population parameters. A Monte Carlo simulation study indicates that the two most commonly used adjustments in meta‐analysis are ineffective in estimating population‐based effect sizes. They frequently overestimate true relationships, sometimes severely. Indeed, simulations show that common adjustments move estimates away from population values in many situations. We suggest an alternative approach to adjustment that produces more accurate and conservative estimates. We apply our approach to examples in the literature and call into question estimates and interpretations in a large body of meta‐analyses based on aggregated group‐mean variables.