Understanding what most people really like or want is not something you can figure out by just talking to your friends. Polling the general public requires a practiced, scientific approach. And a lot of people see it as a crucial part of our democracy.
“Oftentimes what elites, our elected officials, care about can often be very different from what the public cares about,” said Marisa Abrajano, a political science professor at UC San Diego who uses surveys in her research.
“So we want to make sure, as a representative democracy, that legislators are being responsive to the needs of their constituents and this is where public opinion plays a valuable role in understanding those needs.”
An opinion poll is a set of interviews with a finite number of people, chosen at random, whose views can represent a much larger group of people. Likely voters, for instance.
The invention of the telephone made the job a lot easier. But today that’s not the only way to reach people. UC San Diego Political scientist Thad Kousser hearkens back to telephone polling in what he calls the “old days.”
“Now, in the old days, to do that you’d use random digit dialers to pull a random subsample of the phone numbers. But who answers their phone now? People have multiple phones. Maybe a few cell phones in a family but no landline” Kousser said.
In other words, welcome to the 21st century of telecommunications.
People do still call people, for surveys or anything else. But they also text. They email. They communicate through social media.
With so many ways of contacting people, what do you do? Maybe you just have to use all the tools and devices the public has become accustomed to.
“Currently the best way to do polling and survey research is to do a multi-mode approach,” said John Nienstedt, president of Competitive Edge Research and Communication.
Nienstedt said different people communicate in different ways, depending on things like their age and levels of education. So if you are not using multiple ways of reaching out, you’re not reaching a random, representative public.
“You have to understand the sample methodology so they all balance out. You don’t want to have 90% of your surveys done by a text or 80% of your surveys done by a landline,” Nienstedt said. “You have to have the mix and the proportion that would be reflected in the population.”
Professional pollsters like Nienstedt sometimes start with lists of people from sample providers. Those are companies — sometimes they’re politically partisan — that take raw data and create lists of people with some demographic information attached.
People who take the time to be in a survey may get an incentive. Gift cards are a common one. Some people are more accessible than others.
“Typically older folks who’ve voted a lot are reachable. Renters, younger people, very busy people are less reachable,” Nienstedt said.
Every group of respondents will somehow fall short of fully representing the general public. That’s why pollsters use a process called weighting. If you don’t have enough of a certain group of people in your sample, you give them, and their opinions, a greater statistical weight when you sum up the results.
Kousser said statistical weighting can expand your polling options. Let’s say you have a group of people who want to be part of a survey and they’re willing to take it online.
“We know about their demographics, their education level, their race and ethnicity. Where they live. Their age. And we can pull people from those buckets,” Kousser said. “And if we have too many people in one bucket we can down-weight their survey responses to get a fairly good estimate that’s representative of the population as a whole.”
Back in the “old days” of polling, when almost everyone had a telephone landline, getting a random sample was a lot simpler. But there were downsides.
Nienstedt remembers the days prior to the deregulation of the telephone industry.
“For instance, just to take us back to the 70s you didn’t have that breakup of the phone companies so calling long-distance was $1.25 a minute. It was expensive to do!” he said. “And that is one of the things that really created the polling industry today was that breakup, and it became cheap to call interstate.”
The hard work of pursuing a random opinion sample today means you can’t rely on people to opt in to a survey with no incentives and no encouragement.
Nienstedt said there’s a big difference between just putting a survey on a website and hoping someone will fill it out, and proactively pursuing every respondent and learning who they are.
Kousser said in the end, all respondents have to decide whether they want to be part of an opinion poll.
“Every survey is an opt in survey, right? There’s no law that says they have to pick up the phone when someone calls you or you have to respond to that email.” he said.
And that is why even the best survey is bound to get it a little bit wrong.
