Regardless of individual political beliefs, one has to agree that 2016 was a bad year for the polling industry. The election outcome caught everyone by surprise and I was no exception. I am a big fan of data aggregation and have a lot of unanswered questions.
How could an entire industry be so wrong? Predicting the outcome is their business and they have successfully done that for decades. This article is my attempt to answer some of these questions and it also explores the polling industry and its evolution and its struggle to stay relevant in this age of new media.
According to dictionary.com, a poll is: a sampling or collection of opinions on a subject, taken from either a selected or a random group of persons, as for the purpose of analysis.
While discussing opinion polls, it would be good to look at the company Gallup. The company became known for its public opinion polls conducted worldwide. It has historically measured and tracked the public's attitudes (through public opinion polls) concerning political, social, and economic issues, including sensitive or controversial subjects (Wikipedia.com). Gallup has established a methodology to conduct its opinion polls. Here is an excerpt from their website:
Gallup Daily tracking is divided into two surveys: Gallup U.S. Daily and the Gallup-Healthways Well-Being Index. Gallup interviews 500 U.S. adults for each survey, 350 days per year with minimum quotas of 30% landlines and 70% cellphones. These large samples allow Gallup to examine extensive demographic breaks and unique cross-tabulations of the daily measures.
Given that this company is reputable and has an established methodology, the next question would be, how accurate is their prediction based on their data. Apparently, Gallup Polls correctly predicted the winner of the presidential election with the notable exceptions of the 1948 and 1976, from 1936 to 2008 (Wikipedia.com).
So, reputable companies who have established methodologies (on opinion polls and election predictions) perfected over decades, were wrong about the election outcome on November 9th. The question that comes to mind is ‘what was different this time.’ Since the election, there has been a lot of back and forth blame between media and the polling companies, but the answer may not be warranting a high degree of analysis. I found an article on scienceblog.com and I think, they have articulated the issue clearly (Read the full post here - https://scienceblog.com/490276/big-data-future-election-polling/).
People are likely to be more honest when telling friends rather than answering polls. It is scary how accurate prediction can be done by analyzing social media.
Image Source: Scienceblog.com
Going back to the why the pollsters were wrong, here are a few reasons:
- Misrepresenting and misreporting of polls in a situation of high uncertainty (https://fivethirtyeight.com)
- Confidence interval was not factored into the calculations (https://fivethirtyeight.com)
- Popular vote versus electoral college vote (https://fivethirtyeight.com)
- Non-response bias (gallup.com)
While each of these reasons is valid, the fact that seasoned professionals collectively could not foresee this outcome does raise some questions. It might be time to revise the methodology and adjust the outcome based on uncertainties.
I remember working on an assignment in my market research class where we were trying to (accurately) predict a box office hit though the twitter mentions. This is a different world. People don’t just rely on polls and surveys to voice their opinions. There are other outlets. So, can the polling industry survive? The answer is YES! I have a few suggestions:
- Educate the press on confidence intervals, high margin of error and call them out when they misreport the predictions
- Uncertainty adjustment (for instance there were lot of undecided voters in the 2016 election)
- Update sample cross section frequently (it is possible the polls were conducted among a cross-section of people who were different from the actual voters)
- Explore a different discipline for measuring emotions (via social media) and factor that into the methodology
I am a marketing professional with more than 10 years’ experience. I currently work as a Design QC Specialist contractor with Chevron Creative Studio. I have worked in industries such as non-profit, healthcare, oil and gas, real estate, technology, education and consumer goods.
I recently earned my MBA from the University of Houston, BAUER School of Business with a focus in Marketing Analysis. Prior to that, I earned an Associates in Graphic Design from the Art Institute of Houston and Masters in English Literature from Gujarat University, Ahmedabad, India.
I was born and raised in India but now consider Houston my home. I am Membership Officer with Prospanica-Houston chapter and Director of Special Events-Marketing with American Marketing Association-Houston chapter.
Connect with Rinki on LinkedIn