In mid-2007, Princeton Survey Research Associates conducted a major survey for the Pew Internet Project and the University of Illinois-Urbana-Champaign about how people solve certain kinds of problems (Information Searches that Solve Problems, 12/30/2007). Funding for this survey was provided by a federal agency called the Institute for Museum and Library Services. All surveys supported by the federal government are reviewed by the White House's Office of Management and Budget. In the process of evaluating our survey, OMB officials asked if we could do additional work to see if non-respondents to the initial poll were different from those who did respond. The following article excerpts the major findings of that additional analysis.
Summary of findings
In the past two decades, the research profession has faced increasing numbers of Americans who fail to complete interviews. The levels of such non-response raise questions about the representativeness and validity of surveys and the data they provide. The additional analysis produced at the request of the White House's Office of Management and Budget (OMB) had three elements.
First, the interviewing effort on a sub-sample of the telephone numbers generated through computerized Random Digit Dialing (RDD) methods was doubled. RDD is the practice of pollsters to generate results from a random, representative subset of a population. The original survey design called for a maximum of 10 calls to each number. We selected 1,500 of the phone numbers that did not yield complete survey interviews after 10 calls and phoned them at least 20 times. The results of that extra effort were compared with the results of the standard 10-call effort.
Second, an analysis was conducted of survey results from the base 10-call design by segmenting interviews by the amount of effort actually required to get results.
Third, the total sample of telephone numbers and the subset that provided completed interviews using the 10-call design were analyzed to determine what kinds of communities are under- and over-represented in completed interviews.
The results of the analysis found:
Read the complete survey results How Different Are People Who Don't Respond to Pollsters? on the Pew Research Center Web site.