Part 2 of 3: There’s more to the fine print than ‘Margin of Error’
Editor’s Note: In a battleground state like Wisconsin, political polling is going to become as ubiquitous as cows in the fields. So it’s important to know about the fine print, even in the most professionally designed survey. We asked political writer and Progressive Magazine contributor Jud Lounsbury to explain why he frequently takes issue with the Marquette University Law School poll, as an example.
The Hill called the October poll by the Marquette University Law School Democrats’ “most alarming” indicator that their impeachment effort was going south. AP ominously compared it to the Wisconsin 2012 recall of Scott Walker. And conservative radio host Rush Limbaugh crowed, “In the pivotal swing state of Wisconsin, Donald Trump leads the top four Democratic presidential contenders by three to eight points. The significance is that the same poll released in October had Trump trailing all those candidates badly.”
Indeed, not only had Trump flipped the script on his Democratic opponents, there had also been a doubling of the gap between those who oppose impeachment over those who supported it. Even Democratic governor Tony Evers was caught up in this apparent conservative tidal wave, his approval rating gap plummeted from +14% to +5% approval over disapproval.
And yet, whatever force that swayed Wisconsin voters’ collective psyche left as quickly as it appeared: The January poll, had Evers approval rating back in double digits (+11), Biden beating Trump 49%-45%, and the approve/disapprove numbers for impeachment was now at only 44%-49%.
In an analysis of the poll’s samples from October to November, the single biggest change was in the proportions of conservatives, moderates and liberals. From October to November, the percentage of conservatives in the poll increased by 4 points (from 38 percent to 42 percent of the sample), while the percent of liberals decreased by 3 points (25 down to 22 percent). In addition, the percentage of those that described themselves as “very conservative” increased from 6% to 10% over the same time period.
Compare these numbers to the exit polling on how Wisconsin voted in the 2016 presidential election when the number of self-described conservatives was only 34 percent, not 38 and certainly not 42. The liberal share was 25 percent. And a full 40 percent labeled themselves as moderates in 2016, compared to only 32 percent in both the October and November Marquette polls.
I contend it is most prudent to use the 2016 Wisconsin exit polls as the touchstone –even though it was the first time a Republican won the state since 1984– as it does represent the last presidential election as well as, arguably, the worst-case scenario for Democratic voter turnout in 2020.
I am not at all saying the Marquette polls are bad. To Charles Franklin’s credit, he releases hundreds of pages with each poll showing the demographics of the poll sample that made the poll. This allows us to spot any oversampling and undersampling that might have occurred. He also releases their methodology, which shows the actual poll sample tends to be mostly retirees who are more likely to be home and answer the phone– 56% in the last poll were over age 60. So, the poll is “weighted” by shrinking down the retirees’ numbers and inflating the numbers of the younger groups to match actual demographics of the electorate. This inflating of the smaller groups is where problems can occur– and in the case of the mysterious November poll, it’s likely what happened: From October to November the youngest age group (and smallest actual sample) varied wildly, while the oldest age group (large actual sample) held steady from one month to the next.
Looking under the hood, I have often found significant demographic shifts in the poll sample from one survey to the next, most often resulting in a higher level of Republican-leaning voters. This does not simply involve those who self-identify with this label or that. It also includes occasional undersampling of minority voters, young voters, low income voters and urban voters, as well as an oversampling of voters without a college degree, older voters, non-union voters and rural voters.
The bottomline: We have the benefit of knowing the demographics of voters in previous Wisconsin elections, and we should use those numbers to judge the validity of polls. If a poll doesn’t mirror the electorate it’s attempting to gauge, it’s not a good poll.
NEXT: Differences in recent polling from Quinnipiac University and the University of Wisconsin Election Research Center.