What You Should Know About Polling
Now that Labor Day has passed, it is the home stretch of the election season. A large amount of attention will be directed towards polling and its results from now until Election Day. This shift occurs for good reason, as both conventions are over , both running mates have been announced, and most voters have already made their decision on who they will vote for. Thousands of polls have already been conducted from countless different media outlets, universities, and private polling firms. The sheer volume of polls can be overwhelming; not to mention it can be very difficult to decipher how accurate a poll may be. It is essential for individuals to understand polling so they are not misled by polls they see, and can determine for themselves an accurate picture of the race.
Many people in 2020 are dismissive of polls due to the dissonance between the polling results and the victor of the 2016 election. Hillary Clinton was favored in many swing state national polls, but ended up losing the general election to Donald Trump. Despite the results of the election, the polls were actually not far off. The RealClearPolitics national popular vote average had Clinton up 3.3 points before the election. She won the popular vote by 2 points, so polls were only off by 1.3 points. Polling error in 2016 mainly occurred at the state level. Polling error occurred in states Trump won like Michigan, Wisconsin, and Pennsylvania. Trump won all these states by less than 1%, but Clinton was favored by at least 4 points in all three of these states, according to 270towin.com polling averages. Specifically in the midwestern states Trump was underrated in, there was an unprecedented break of undecided and third party voters for Trump. The moral of the story is, while there was polling error in state polls in 2016, polls are still accurate and should not be disregarded due to the outcome of the 2016 election.
The most important thing about conducting an accurate poll is its methodology. Two main aspects include how it is executed and who is polled. Polls can be conducted via telephone, online, in person, and robocalls. Large media outlets such as CNN and Fox News conduct their polls as live calls. These types of polls are effective, but are becoming more difficult to conduct as Americans are becoming increasingly unlikely to answer calls from unknown numbers. According to Pew Research Center, response rates for these types of polls have dropped 30% since 1997 to only 6%. Low response rates make obtaining a representative sample challenging. To account for rising difficulty in conducting live caller polls, pollsters have turned to internet surveys. In order to ensure a random sample, pollsters such as the Associated Press use lists of physical mailing addresses to recruit people to take the online surveys. However, similar to live calling, recruiting a random sample is difficult. To maximize the efficacy of their efforts, pollsters are using “probability-based online pools”, which essentially are groups of people that take multiple surveys over a period of time, instead of recruiting just for a single poll. These types of polls are useful to see how public opinion has changed over time. Ipsos, USC-Dornsife, and Pew Research all employ this approach. Finally there are “opt-in” surveys. These pollsters advertise their surveys on websites, and record the results from whoever is willing to take the survey. These polls are also called “non probability surveys”, because they do not use random or probability sampling design. Morning Consult, YouGov, and SurveyMonkey conduct these types of polls. Compared to live caller and random sample online polls, opt-in surveys are off by an average of 6-10%. Opt-in surveys are especially inaccurate when polling hard to reach groups, such as the Hispanic community and people under 30 years old.
Another key factor that may explain the polling error that resulted in Clinton being favored in 2016 was the failure to adjust for the overrepresentation of college graduates. Pollsters can remedy this by weighting their polls for education. Since 2016, more polls have executed this strategy in an effort to reflect demographics more accurately. According to Pew Research Center, voters who have a four-year degree are more likely to answer surveys and more likely to vote Democratic. Therefore, since it is extremely likely that a poll will have a high and therefore unrepresentative amount of college graduates participating, it is important to evaluate the poll by its weighted sample. Ideally, the sample can be weighted by high school education or less, share with some college, share with a four year degree, and share with a graduate degree to give the most accurate representation. By state, the distribution of voter education level has remained constant over the past few elections, making weighing by education worthwhile for pollsters.
After taking into account how the poll is executed, it is important to look at who is being polled. Polls are broken down into registered voters (RV) and likely voters (LV). Registered voter polls are conducted from the group of individuals who are registered to vote in their precinct. Likely voter polls are conducted from people who seem likely to vote based on the criteria of the pollster. Questions such as “thought given to the election” and “how often do you vote” are often asked for the pollster to decide if the participant is a “likely voter” or not. See Gallup’s criteria here. In presidential election years from 1992 to 2012, the average bias in likely voter polls was 0.7 points Republican, while the median of the same polls had no bias whatsoever. Registered polls on the other hand, tend to be biased towards Democrats. From 1992 to 2012, the average bias of registered voter polls had a 1.1 bias towards Democrats, while the median had a 2.0 Democratic bias.
Finally, it is important to note who is conducting the poll. Opt-in polls are relatively cheap to conduct, and many partisan firms have entered the polling scene using this method. The objective of partisan polls is often to push their own agenda to shape public opinion, using methodology that favors their candidate. This often means executing “push polls'' or polls that are worded in a way to attract answers the pollster wants, or not sampling a truly representative sample. Individuals can be deceived by the results of these polls. Viewing polls with quality methodology is essential to understanding the true state of the race. FiveThirtyEight’s pollster ratings is a great place to start to find high quality polls. Also, looking at polling averages, such as the ones provided by RealClearPolitics, can help prospective voters understand the state of the race and recent trends. Voters should be prepared to see numerous polls everyday until election day, and by understanding how the polls were executed, who was surveyed, and who conducted the polls, they will have a better idea what they mean for what could happen on November 3rd.
Works Cited:
Cohn, Nate. “A 2016 Review: Why Key State Polls Were Wrong About Trump.” The New York Times, The New York Times, 31 May 2017, www.nytimes.com/2017/05/31/upshot/a-2016-review-why-key-state-polls-were-wrong-about-trump.html.
Deane, Claudia, et al. “Election 2020 Polling Field Guide.” Pew Research Center Methods, Pew Research Center, 30 Dec. 2019, www.pewresearch.org/methods/2019/11/19/a-field-guide-to-polling-election-2020-edition/.
“Election 2016 - General Election: Trump vs. Clinton vs. Johnson vs. Stein.” RealClearPolitics, www.realclearpolitics.com/epolls/2016/president/us/general_election_trump_vs_clinton_vs_johnson_vs_stein-5952.html.
Gallup. “Understanding Gallup's Likely Voter Procedures for Presidential Elections.” Gallup.com, Gallup, 20 Oct. 2014, news.gallup.com/poll/111268/How-Gallups-likely-voter-models-work.aspx.
Gallup. “What Is the Difference between Registered Voters and Likely Voters?” Gallup.com, Gallup, 20 Oct. 2014, news.gallup.com/poll/110287/what-difference-between-registered-voters-likely-voters.aspx.
Hatley, Nick, and Courtney Kennedy. “State Election Polls and Weighting Factors.” Pew Research Center Methods, Pew Research Center, 16 Sept. 2020, www.pewresearch.org/methods/2020/08/18/a-resource-for-state-preelection-polling/.
Kennedy, Courtney. “Key Things to Know about Election Polling in the United States.” Pew Research Center, Pew Research Center, 20 Aug. 2020, www.pewresearch.org/fact-tank/2020/08/05/key-things-to-know-about-election-polling-in-the-united-states/.
NateSilver538. “Pollster Ratings.” FiveThirtyEight, 19 May 2020, projects.fivethirtyeight.com/pollster-ratings/.
NBC News/Wall Street Journal Survey. “Study #200356.” Hart Research Associates/Public Opinion Strategies/Vision Strategy & Insights, July 2020.
Palmer, Brian. “Why Are There Democratic and Republican Pollsters?” Slate Magazine, Slate, 23 Apr. 2012, slate.com/news-and-politics/2012/04/partisan-polling-why-are-there-democratic-and-republican-pollsters.html.
Silver, Nate. “Registered Voter Polls Will (Usually) Overrate Democrats.” FiveThirtyEight, FiveThirtyEight, 9 Sept. 2014, fivethirtyeight.com/features/registered-voter-polls-will-usually-overrate-democrats/.
Edited by: Monica Wallace
