Perils of Polling in Election '08

Report Read time:
Perils of Polling in Election '08

The analysis of total survey error has evolved over many decades to consider a wide variety of potential threats, including concerns about the contribution of both bias and variance, and an attention to errors of both observation and non-observation (Groves 1989). The validity of public opinion polling in the presidential election of 2008 was thought to be seriously imperiled by a wide range of these potential errors. Among these were coverage error due to the growth of the wireless-only population, nonresponse error potentially caused by differential nonresponse among Republicans and racially conservative voters, and measurement error potentially resulting from racially-related understatement of support for the Republican candidate and greater-than-usual difficulties in forecasting turnout and identifying likely voters.

Despite these obstacles, polls performed very well, with 8 of 17 national polls predicting the final margin in the presidential election within one percentage point and most of the others coming within three points. Both at the national and state levels, the accuracy of the polls matched or exceeded that of 2004, which was itself a good year for the polls. The performance of election polls is no mere trophy for the polling community, for the credibility of the entire survey research profession depends to a great degree on how election polls match the objective standard of election outcomes. The consequences of a poor performance were dramatically demonstrated in the reaction to the primary polls' inaccurate prediction that Barack Obama would win in New Hampshire, portrayed as one of polling's great failures in the modern political era ((AAPOR 2009 (PDF)).

The Pew Research Center examines the challenges of potential coverage bias from excluding cell phones and potential measurement and non-response bias due to race in detail using data from a wide range of sources, including a summary analysis of state and national pre-election polls, six telephone surveys conducted among both landline and cell phone samples, and a comparison of a survey conducted by landline with reluctant and elusive respondents with a survey conducted at the same time with a fresh sample using standard methodology. The conclusion is that some of the threats were very real but overcome by the techniques normally employed in surveys to address potential bias from various sources of error, while other threats turned out to be less serious than some anticipated.

Read the full report Perils of Polling in Election '08 on the Pew Research Center's Web site.

America’s Overdose Crisis
America’s Overdose Crisis

America’s Overdose Crisis

Sign up for our five-email course explaining the overdose crisis in America, the state of treatment access, and ways to improve care

Sign up
Quick View
America’s Overdose Crisis

America’s Overdose Crisis

Sign up for our five-email course explaining the overdose crisis in America, the state of treatment access, and ways to improve care

Sign up

MORE FROM PEW

A presentation being given by a businesswoman to a group of participants at a business conference. Four Plan Features Can Boost 'Retirement Readiness'
A large group of people in formal or traditional attire stands together on a stage, with a digital background depicting marine life and the large text “WTO Agreement on Fisheries Subsidies.” WTO Needs to Redouble Its Effort on Fishing Subsidies
A man in a winter hat and sunglasses kneels on a rocky shoreline in front of a campfire, holding a metal mug with gloved hands. Behind him, in the water, a motorboat is moored to the shore. Cliffs rise on both sides of the water. Why Newfoundland’s South Coast Fjords Are Sacred
An illustration of a couple holding hands walking away from an outline of houses behind them Pathways to Homeownership: Housing in America
Composite image of modern city network communication concept

Learn the Basics of Broadband from Our Limited Series

Sign up for our four-week email course on Broadband Basics
Quick View

How does broadband internet reach our homes, phones, and tablets? What kind of infrastructure connects us all together? What are the major barriers to broadband access for American communities?

Pills illustration
Pills illustration

What Is Antibiotic Resistance—and How Can We Fight It?

Sign up for our four-week email series The Race Against Resistance.
Quick View
Pills illustration

Antibiotic-resistant bacteria, also known as “superbugs,” are a major threat to modern medicine. But how does resistance work, and what can we do to slow the spread? Read personal stories, expert accounts, and more for the answers to those questions in our four-week email series: Slowing Superbugs.