How (Not) to Estimate the Design Effect of a Complex Sampling Scheme: A Case Study of the Polish Section of the European Social Survey, Round 5
Keywords:European Social Survey
effect of unequal sampling probabilities
MetadataShow full item record
Citation:Ask: Research and Methods. Volume 22, Issue 1 (2013), pp. 55-77
Design effect (DEFF) is a measure used to assess the effectiveness of a particular sampling scheme. Even though its definition is remarkably simple (cf. Kish 1965: 258), its practical implementation turns out to be problematic. Researchers therefore usually simplify the estimation of DEFF by independently determining the values of three components, namely, the clustering effect (DEFFc), the stratification effect (DEFFs) and the effect of unequal sampling probabilities (DEFFp) and by multiplying these partial measures to obtain a measure of overall effect. However, the validity of such a simplified version depends on strict formal requirements which are met only in a few sampling schemes. The subject of the analysis presented here is the sampling scheme in the Polish section of round 5of the European Social Survey (ESS). It will be shown that the method of DEFF estimation applied by the Polish coordinators of the project, which is compatible with the methodological recommendations of ESS (cf. Lynn et al. 2007: 114),does not satisfy the formal criteria that would validate its use. The author proposes two other ways of estimating the size of DEFF (cf. Gabler et al. 2006: 116-117) appropriate for the sampling scheme in ESS5-PL. Empirical analyses indicate that the use of the simplified procedure of DEFF prediction leads to significant underestimation of variance inflation in the sample design of ESS5-PL and, in turn, to overestimation of effective sample size.
Rights:This item may be protected by copyright, and is made available here for research and educational purposes. The user is responsible for making a final determination of copyright status. If copyright protection applies, permission must be obtained from the copyright holder to reuse, publish, or reproduce the object beyond the bounds of Fair Use or other exemptions to the law. We are eager to hear from any copyright holders who are not properly identified.
Items in Knowledge Bank are protected by copyright, with all rights reserved, unless otherwise indicated.