The First Straw, and Hopefully Not the Last

By the time Election Day rolls around, Americans have been inundated by polls. For months leading up to November, those of us who follow campaigns closely watch the numbers creep up and slink back down again. They can be mesmerizing. They can also be wrong.

Polls sometimes overstate the influence of one group and under-survey another. They fail to keep up with changing technology and adjust their methods to match. They all but promise us one outcome, and we get another. But still, we pay close attention, studying the numbers state by state, demographic by demographic, for any hint of changing attitudes.

Polls, for all their faults, are all we’ve got. Unlike willing patrons of tarot card dealers or palm readers, we have no choice but to put our faith in these tiny snapshots of the American electorate and trust they are both representative and accurate.

The country didn’t always have polls with which to drive itself crazy. The “earliest counterpart of modern opinion surveys,” in the words of George Gallup (yes, that Gallup), was born in 1824. The 1824 election was unique in many ways, but mainly because the four candidates running were all members of the same party – the Democratic-Republicans. Given the chaos, voters began to take ‘straw polls’ to see who their fellow countrymen were voting for.

These polls were taken at militia assemblies, grand juries, Fourth of July celebrations, and just about any other gathering that was comprised of white males. Named after the thin plant, straw polls offered the country a glimpse into which way the metaphorical wind blew.

“I think people took this as interesting and informative, but not in any sense definitive,” said Tom W. Smith, who wrote about the 1824 straw polls for the Public Opinion Quarterly.

Historians know of these first polls because contemporary newspapers reported on them. As the 19th century chugged along, magazines and newspapers began conducting their own polls to offer their readers insight into what their neighbors thought. Still, the surveys remained regional and were not framed as national election predictors but rather glimpses into local opinion.

This changed in 1916 when The Literary Digest introduced a national presidential poll that correctly predicted the victor of presidential elections five times straight. The magazine’s methods were flawed, however, and the streak ended in 1936 when George Gallup correctly predicted Franklin D. Roosevelt would win reelection, and The Literary Digest insisted Roosevelt would lose to Kansas Gov. Alf Landon in a landslide.

It was this substantial miss by The Literary Digest that caused the magazine to cease publication and prompted American pollster and statistician Archibald Crossley to wonder in a 1937 essay whether it was “possible to sample public opinion sufficiently accurately to forecast an election, particularly a close one?”

Gallup’s polling firm argued yes, it is possible if one simply uses quota-controlled methods and surveys small sample sizes. Gallup’s organization continued to conduct polls on public opinions and preferences using modern statistical methods and, for the most part, proved the efficacy of the new way. One infamous error, however, occurred in 1948.

Gallup’s company – along with other polling firms – predicted that New York Gov. Thomas E. Dewey would beat incumbent President Harry S. Truman. All the polls were in agreement, giving the Chicago Daily Tribune enough confidence to plaster “Dewey Defeats Truman” on the front page of its paper the day after the election. Truman proved polls are not always to be trusted, immortalized in a photo of the victorious president holding up an inaccurate newspaper headline.

One issue with those polls based on small sample sizes is that “they are too small to reliably detect the relatively small day-to-day or week-to-week movements in voter sentiment we would expect to occur over an election campaign,” in the words of political scientist Simon Jackman. Enter polling aggregation.

Individual polls, being a product of the fickle discipline of statistics, are subject to random sampling error and hundreds of potential biases by any firm’s particular polling methodology. Recognizing this, several online poll aggregators, including RealClearPolitics and FiveThirtyEight.com, cropped up in the early 21st century.

Polling aggregation improves the precision of forecasts, but there have still been “surprise” victories over the years – notably in 2016 when Donald Trump shocked the world and beat former Secretary of State Hillary Clinton for the presidency. In the lead-up to the 2016 election, polls indicated that Clinton ought to begin writing her inauguration speech. But Trump carried several key battleground states, including some states Democrats thought were in the bag.

Pollsters typically weight for characteristics such as age, race, or gender to ensure that polling samples match up with census data. In 2016, pollsters failed to account for a person’s education level and vastly underestimated Trump’s support among white non-college-educated voters, a group he won by a margin of more than two to one.

In 2016, 88% of national polls overstated the Democratic candidate’s support among voters. Four years later, when pollsters were thought to have fortified their weak points, 93% of national polls overstated the Democratic candidate’s support. The two elections raised questions about the state of public opinion polling, and America reiterated Crossley’s 1937 question: “Is it possible to sample public opinion sufficiently accurately to forecast an election, particularly a close one?”

Scott Keeter, senior survey adviser at the Pew Research Center, said there was “understandable surprise on the part of the people” following the two elections. Of pollsters, Keeter said they got a “wake-up call.”

Since 2016, 61% of pollsters have changed their approach to polling, according to a study by Pew Research. Fewer pollsters rely solely on live phone surveys, while the methods of online polling and text messaging are growing rapidly. Most pollsters now use multiple methods to grab the most representative sample possible and avoid mistakes of the past.

Like all American innovations, polling continues to improve. It will never be perfect, but it will continue to give us glimpses into which way the wind blows.