Many state survey results show a connection. So are they connected because of voters or opinion polls? – NBC New York

Recent polls in seven major swing states show a surprisingly tight presidential race, with 124 of the last 321 polls taken in those states (nearly 39%) showing margins of 1 percentage point or less.

In fact, state polls show not just a surprisingly tight race, but an incredibly tight race. Even in a truly close election, the randomness inherent in polling will lead to more varied and less clustered results—unless state polls and polling averages are artificially close due to the decisions pollsters make.

The survey results depend on the opinions of voters and the decisions of sociologists. Decisions about how to weight poll results to match the expected composition of the electorate. You can shift survey results up to 8 points. This is true even if pollsters make perfectly reasonable decisions about how to weight their survey data, as polling researchers have been forced to consider new methods and ideas for weighting and addressing falling response rates following survey errors in 2016 and 2020.

But the fact that so many polls report the same metrics and results raises a troubling possibility: some pollsters are making adjustments in such similar ways that those choices result in results being lumped together, creating a potential illusion of certainty—or that some pollsters are even turning to the results of others in order to be guided by them (i.e. “herd”). If so, the artificial similarity of the polls could create a false impression that may not show up on Election Day. We may well be in for a very close election. But there is also a significant chance that one candidate or another could prevail in every swing state and win the presidency fairly comfortably, at least compared to the evenly balanced picture in the polls.

In a perfect polling world, what should we see due to randomness?

In a poll-perfect world—a researcher’s paradise in which every voter can be contacted and every voter contacted responds—we can use mathematics to calculate how much difference there should be as voters are randomly selected to participate in a survey .

If the race in this world were truly 50%-50%, not all polls would produce results that split 50%-50%. Imagine if pollsters in this world conducted 100 identical polls among 863 randomly selected voters (that’s the average sample size of polls in swing states this year). The results of 95 of these polls show candidates receiving support somewhere between 46.7% and 53.3% – although we know in this imaginary world that the actual race is 50%. The remaining five surveys will show candidates earning even more or less outside this range.

This variation is known as the “bias” in the survey—i.e. the extent to which randomly selected voters who always respond can influence a candidate’s rating in a poll.

Because each candidate’s support varies randomly, these polls predict margins in close races ranging from -6.6 to +6.6 for 95 of the 100 polls (with even larger margins in the other five).

It’s important to emphasize that the range of difference we can expect in a close race (and in a perfect polling world) is much larger than the difference in swing states in 2020. Even in ideal voting circumstancesit is difficult, if not impossible, to conduct a poll to determine who is leading in a tight race. And that’s perhaps the lower end of what we should see in the messier real world, where polls vary in how respondents are selected, contacted and weighted to match the electorate pollsters think will turn out in 2024.

We can also calculate what proportion of 863-person polls we should expect to show a different margin in a truly close race. Rounded to the nearest percentage point, about 11% of polls in a close race should show a tie.

This means that almost 9 out of 10 peer-race polls should not actually show a tie due to randomness and margin of error.

About 32% of polls should have a difference of 1 point or closer, 55% should have a difference of 2 points or closer, and 69% should have a difference of 3 points or closer. Even in a 50-50 race, about 10% of the polls should have a difference of more than 5 points due to their inherent randomness – almost the same percentage that shows a (rounded) tie!

Given enough polls, the predicted difference should also resemble a normal “bell curve” distribution—with the same number of polls showing either candidate leading.

What are we seeing in swing state polls?

Actual polls of swing states show much smaller differences than the benchmarks we would expect in a perfect polling world. Of 321 polls in seven swing states, only 9 polls (3%) report a difference of more than 5 points. Even if the races were all tied (which they are not), we would still expect to see about 32 of the 321 polls with a difference of more than 5 points due to randomness.

Visualizing how the reported polling difference compares to what we would expect in a perfect polling world provides strong evidence for the “staging” of polling differences in swing states around statewide polling averages. In these 321 state polls, 69 of them (21%) report an exact tie, while 124 polls (39%) report a difference of 1 percentage point or less. Both of these figures are about double what we would expect in an ideal world of polling, where the only source of variation is the random selection of voters who respond.

Josh Clinton/NBC News

Pennsylvania is perhaps the most alarming state. Fully 20 of the 59 polls (34%) show an exact tie, and 26 (44%) show a difference of 1 point or less. the 5 point difference is due to chance, we see only 2 of the 59 Pennsylvania polls (3.3%) with a difference greater than 5 points.

Josh Clinton/NBC News

Even where polling results are not as tightly clustered, such as in Arizona, Michigan and Wisconsin, there are still far more polls than would be expected within the polling average, and too few polls by a wide margin.

What’s happening?

The concentrated difference we see in swing state polls likely reflects one of two possibilities.

One possibility is that pollsters can sometimes adjust survey results that seem “odd” to them by choosing a weighting scheme that produces results that are closer to those of other polls. Risk-averse social scientists seem to have strong incentives to do so. If a pollster does not conduct many polls and cannot be sure that the effects of chance are averaged out, he may fear the reputational and financial costs of getting the wrong result due to chance, since polls are judged on polling accuracy.

A risk-averse pollster with a 5-point lead in a race that he believes is close may decide to “adjust” the results so that they are closer to what other polls are showing, so that their poll is out of bounds. , did not negatively affect its reputation compared to its competitors.

Another, more likely possibility is that some of the tools pollsters are using in 2024 to address the problems of 2020 polls, such as weighting by party affiliation, past voting or other factors, may smooth out differences and reduce differences in reported polling results . The effect of such decisions is small but important because it means that the similarity of polls is determined by the decisions of pollsters, not voters.

And if those assumptions are wrong, and there’s no way to know until after the election, then the risk of potentially significant polling error increases as differences across different polls decrease.

Why is this important

The fact that so many polls in swing states are reporting the same difference in results is problematic because it raises the question of whether the polls in these races are the same because of voters or pollsters. Will 2024 be as close to 2020 because our politics are stable, or will 2024 polls be similar to 2020 only because of the decisions government pollsters make? The fact that the polls appear to be more clustered than we would expect in a perfect polling world raises serious questions about the second scenario.

Published polls and polling averages suggest that the race will be very close and we will likely see a result similar to 2020. Perhaps this is true. It would be great if the polls in 2024 successfully resolved the problems of 2016 and 2020.

But the fact that all polls show such a similar difference does not necessarily make it more likely that this difference reflects the final result. In fact, it raises the possibility that election results could be unexpectedly different from what state polls and polling averages suggest.

This story first appeared on NBCNews.com. More from NBC News: