Be skeptical about weighted survey estimates and inspect the alternatives

Why do I insist that scientific articles must show unweighted numbers as well as estimates based on these two weights: (1) weighted by probability of selection only, and (2) weighted by the sampling weight and the post-stratification factors as well. If you study only (2), but not (1) and the unweighted numbers, you've started to drink some of the Jonestown kool-aid.

Read this article for some insights:

http://www.nytimes.com/2016/10/13/upshot/how-one-19-year-old-illinois-man-is-distorting-national-polling-averages.html?_r=0

If no information is available beyond (2), as in R-DAS, at least ask for an approximation of unweighted numbers, whether subgroup estimates have been reported based on surpassing a minimum unweighted observation count, and ask for a distribution of estimates stratified by quantiles of the available analysis weights. Look for anomalies.

#survey
#tools