Perils of Polling in Election '08

The analysis of total survey error has evolved over many decades to consider a wide variety of potential threats, including concerns about the contribution of both bias and variance, and an attention to errors of both observation and non-observation (Groves 1989). The validity of public opinion polling in the presidential election of 2008 was thought to be seriously imperiled by a wide range of these potential errors. Among these were coverage error due to the growth of the wireless-only population, nonresponse error potentially caused by differential nonresponse among Republicans and racially conservative voters, and measurement error potentially resulting from racially-related understatement of support for the Republican candidate and greater-than-usual difficulties in forecasting turnout and identifying likely voters.

Despite these obstacles, polls performed very well, with 8 of 17 national polls predicting the final margin in the presidential election within one percentage point and most of the others coming within three points. Both at the national and state levels, the accuracy of the polls matched or exceeded that of 2004, which was itself a good year for the polls. The performance of election polls is no mere trophy for the polling community, for the credibility of the entire survey research profession depends to a great degree on how election polls match the objective standard of election outcomes. The consequences of a poor performance were dramatically demonstrated in the reaction to the primary polls' inaccurate prediction that Barack Obama would win in New Hampshire, portrayed as one of polling's great failures in the modern political era ((AAPOR 2009 (PDF)).

The Pew Research Center examines the challenges of potential coverage bias from excluding cell phones and potential measurement and non-response bias due to race in detail using data from a wide range of sources, including a summary analysis of state and national pre-election polls, six telephone surveys conducted among both landline and cell phone samples, and a comparison of a survey conducted by landline with reluctant and elusive respondents with a survey conducted at the same time with a fresh sample using standard methodology. The conclusion is that some of the threats were very real but overcome by the techniques normally employed in surveys to address potential bias from various sources of error, while other threats turned out to be less serious than some anticipated.

Read the full report Perils of Polling in Election '08 on the Pew Research Center's Web site.

Spotlight on Mental Health

Composite image of modern city network communication concept

Learn the Basics of Broadband from Our Limited Series

Sign up for our four-week email course on Broadband Basics

Quick View

How does broadband internet reach our homes, phones, and tablets? What kind of infrastructure connects us all together? What are the major barriers to broadband access for American communities?

Pills illustration
Pills illustration

What Is Antibiotic Resistance—and How Can We Fight It?

Sign up for our four-week email series The Race Against Resistance.

Quick View

Antibiotic-resistant bacteria, also known as “superbugs,” are a major threat to modern medicine. But how does resistance work, and what can we do to slow the spread? Read personal stories, expert accounts, and more for the answers to those questions in our four-week email series: Slowing Superbugs.

Explore Pew’s new and improved
Fiscal 50 interactive

Your state's stats are more accessible than ever with our new and improved Fiscal 50 interactive:

  • Maps, trends, and customizable charts
  • 50-state rankings
  • Analysis of what it all means
  • Shareable graphics and downloadable data
  • Proven fiscal policy strategies

Explore

Welcome to the new Fiscal 50

Key changes include:

  • State pages that help you keep track of trends in your home state and provide national and regional context.
  • Interactive indicator pages with highly customizable and shareable data visualizations.
  • A Budget Threads feature that offers Pew’s read on the latest state fiscal news.

Learn more about the new and improved Fiscal 50.