Polling lab
See how weighting changes a poll.
See how survey weighting works, one step at a time, and how it changes a result.
Synthetic data. Every respondent on this page is made up for teaching. This is not a real poll and says nothing about any election.
Step 1. The sample and the targets
Choose the population shares the weighted sample should match. Sliders are rescaled to add up to 100 percent.
Question: "Do you prefer weekday over weekend voting hours?" (yes or no, made-up answers)
The default shares are rounded and illustrative, close to the pattern in Census ACS tables for U.S. adults but not copied from a specific table. Change them to see what happens.
Step 2. Unweighted and weighted result
Share answering yes, before and after weighting.
Unweighted yes
38.3%
Margin of error ±3.0 points
Weighted yes
41.3%
Margin of error ±3.9 points
Design effect (Kish)
1.68
1.00 means no precision lost
Effective sample size
597
out of 1,000 respondents
Weighting moved the result by 3.0 points. Raking took 4 passes; largest remaining gap to a target is 0.0000 points.
Step 3. The weights
Each respondent gets a weight. 1.0 means counted once, above 1.0 counts for more, below 1.0 for less.
| 0.0 to 0.4 | 155 |
|---|---|
| 0.4 to 0.8 | 405 |
| 0.8 to 1.2 | 202 |
| 1.2 to 1.6 | 73 |
| 1.6 to 2.0 | 73 |
| 2.0 to 2.4 | 18 |
| 2.4 to 2.8 | 43 |
| 2.8 to 3.3 | 8 |
| 3.3 to 3.7 | 6 |
| 3.7 to 4.1 | 4 |
| 4.1 to 4.5 | 4 |
| 4.5 to 4.9 | 6 |
| 4.9 to 5.3 | 0 |
| 5.3 to 5.7 | 0 |
| 5.7 to 6.1 | 0 |
| 6.1 to 6.5 | 3 |
Smallest weight 0.36, largest weight 6.42
Trimming limits any one person's influence and raises the effective sample size, but the weighted shares then drift away from the targets. Watch the table below.
Step 4. Sample share against target share
Raking adjusts the weights until the weighted share in every group matches its target.
| Variable | Group | |||||
|---|---|---|---|---|---|---|
| Age group | 18-29 | 105 | 10.5% | 21.0% | 21.0% | 2.00 |
| Age group | 30-44 | 234 | 23.4% | 25.0% | 25.0% | 1.07 |
| Age group | 45-64 | 404 | 40.4% | 33.0% | 33.0% | 0.82 |
| Age group | 65+ | 257 | 25.7% | 21.0% | 21.0% | 0.82 |
| Education | High school or less | 150 | 15.0% | 37.0% | 37.0% | 2.47 |
| Education | Some college | 246 | 24.6% | 28.0% | 28.0% | 1.14 |
| Education | Bachelor's or more | 604 | 60.4% | 35.0% | 35.0% | 0.58 |
| Region | Northeast | 192 | 19.2% | 17.0% | 17.0% | 0.89 |
| Region | Midwest | 268 | 26.8% | 21.0% | 21.0% | 0.78 |
| Region | South | 307 | 30.7% | 38.0% | 38.0% | 1.24 |
| Region | West | 233 | 23.3% | 24.0% | 24.0% | 1.03 |
11 of 11 rows
How it works, in plain language
- Collect answers. The people who respond are rarely a perfect miniature of the population. Some groups answer more often than others.
- Pick targets. For traits the whole population is known to have, such as age, education and region, choose the shares the sample should show.
- Rake. Start with every weight at 1. Scale the weights so the age shares match the target, then education, then region. Fixing one can disturb another, so repeat the cycle until all three fit.
- Rescale. Weights are scaled so they add up to the number of respondents. An average weight of 1 keeps the sample size easy to read.
- Measure the cost. Unequal weights make an estimate less precise. The Kish design effect is n times the sum of squared weights divided by the squared sum of weights. The effective sample size is n divided by that number, and the margin of error is computed from it.
What can go wrong
- Very large weights. When a group is rare in the sample but common in the target, each member gets a large weight (largest here: 6.42). A few people then carry too much influence and the margin of error grows. Trimming helps but adds its own bias.
- Small cells. No group here has fewer than 50 respondents; try 300 respondents to see small groups. Weights built on a handful of people are unstable, and one unusual answer can move the result.
- Weighting does not fix who refused to answer. Weights only correct for traits we know and match. If the people who decline differ from those who answer in ways the targets do not capture, the weighted result can still be off.
- Targets are choices. Different reasonable targets give different results. Move the sliders to see how much the answer depends on them.