Any staticians in the house -- how is political poling done now?

[NobleSavage]

Traditionally, most campaigns just called a sample of telephone numbers. It seems if you did that now, it would be heavily skewed to older voters. Most of the people I know have canceled their land line and just use a cell phone. Can you legally call a cell phone for a poll? The main reason I cut my land line was because I was constantly getting telemarketer calls. I've never once gotten a unsolicited call on my cell phone.

Poling on the internet seems fraught with problems. So how does Nate Silver do it?

 

[dismal]

If you get the full reports from the big pollsters they will usually have a section on methodology.

One thing they do is readjust the raw polling results to what they think is a more representative sample. For example, if they run a poll and it comes back 45% Dems, 40% Rep, 15% other and they think the world is really 40% Dem, 30% Rep, 30% other they will reweight the raw poll results to what they think is a representative sample.

 

[dismal]

Here's an example for ye:

About the Survey

The analysis in this report is based on telephone interviews conducted June 5-8, 2014 among a national sample of 1,004 adults 18 years of age or older living in the continental United States (503respondents were interviewed on a landline telephone, and 501were interviewed on a cell phone, including 301 who had no landline telephone). The survey was conducted by interviewers at Princeton Data Source and Thoroughbred Research under the direction of Princeton Survey Research Associates International. A combination of landline and cell phone random digit dial samples were used; both samples were provided by Survey Sampling International. Interviews were conducted in English and Spanish. Respondents in the landline sample were selected by randomly asking for the youngest adult male or female who is now at home. Interviews in the cell sample were conducted with the person who answered the phone, if that person was an adult 18 years of age or older. For detailed information about our survey methodology, see: http://people-press.org/methodology/.

The combined landline and cell phone sample are weighted using an iterative technique that matches gender, age, education, race, Hispanic origin and region to parameters from the 2012 Census Bureau’s American Community Survey and population density to parameters from the Decennial Census. The sample also is weighted to match current patterns of telephone status, based on extrapolations from the 2013 National Health Interview Survey. The weighting procedure also accounts for the fact that respondents with both landline and cell phones have a greater probability of being included in the combined sample and adjusts for household size among respondents with a landline phone. Sampling errors and statistical tests of significance take into account the effect of weighting. The following table shows the unweighted sample sizes and the error attributable to sampling that would be expected at the 95% level of confidence for different groups in the survey:

about the survey methodology

Sample sizes and sampling errors for other subgroups are available upon request.

In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.

http://www.people-press.org/2014/06/09/public-has-doubts-about-bergdahl-prisoner-exchange/2/

 

[Jimmy Higgins]

Are you asking about in-house Republican polling or actual polling?

 

[NobleSavage]

actual polling.

 

[Jimmy Higgins]

actual polling.

The model what the expected voting demographics will be. They contact a bunch of people on the phone to ask them questions and ask for key demographics of the people. They then weight their sample responses to match the demographics they expect.

Republicans do it exactly the same way, except they assume a roughly 80% Republican to 20% Democrat turnout.

 

[Nice Squirrel]

Are you asking about in-house Republican Rasmussen polling or actual polling?

FIFY

I don't know what they are doing wrong.

 

[NobleSavage]

Is there anyway a quality poll could be done on the internet?

 

[dismal]

Is there anyway a quality poll could be done on the internet?

It's not going to be a representative group. Even if you found someway to avoid people flooding in - like you had a randomized way of finding them instead of them finding you.

Perhaps you could find a representative sample of people who answer internet polls. Maybe you can learn over time how to make adjustments to get a sample that is representative of whatever you're looking for.

 

[Jimmy Higgins]

Is there anyway a quality poll could be done on the internet?

Yes. It just hasn't been figured out yet.

 

[NobleSavage]

Is there anyway a quality poll could be done on the internet?

It's not going to be a representative group. Even if you found someway to avoid people flooding in - like you had a randomized way of finding them instead of them finding you.

Perhaps you could find a representative sample of people who answer internet polls. Maybe you can learn over time how to make adjustments to get a sample that is representative of whatever you're looking for.

Is there anyway a quality poll could be done on the internet?

Yes. It just hasn't been figured out yet.

What if a company did it like Nielsen does for TV? Nilson gives like 25K people a box which monitors their TV viewing habits. I think they get some kind of incentive. So maybe a company could give free internet access in exchange for answering poll questions.

I bet Google, 3rd pary ad servers, Facebook, etc. already have some good data just by viewing behavior.

 

[dismal]

Is there anyway a quality poll could be done on the internet?

Yes. It just hasn't been figured out yet.

Most people don't even try. They just pop a poll up on their website and produce wildly unrepresentative samples.

- - - Updated - - -

It's not going to be a representative group. Even if you found someway to avoid people flooding in - like you had a randomized way of finding them instead of them finding you.

Perhaps you could find a representative sample of people who answer internet polls. Maybe you can learn over time how to make adjustments to get a sample that is representative of whatever you're looking for.

Is there anyway a quality poll could be done on the internet?

Yes. It just hasn't been figured out yet.

What if a company did it like Nielsen does for TV? Nilson gives like 25K people a box which monitors their TV viewing habits. I think they get some kind of incentive. So maybe a company could give free internet access in exchange for answering poll questions.

I bet Google, 3rd pary ad servers, Facebook, etc. already have some good data just by viewing behavior.

It depends what you're trying to do. If you're trying to sample facebook users you can probably get a good poll of them on facebook.

If you're trying to sample for an election you will likely get a wildly unrepresentative sample, unless you happen to be a site that attracts a representative sample and no one tells everyone at some other site to go vote in your poll.

 

[Emily Lake]

actual polling.

The model what the expected voting demographics will be. They contact a bunch of people on the phone to ask them questions and ask for key demographics of the people. They then weight their sample responses to match the demographics they expect.

As said, they use representative weighting so that the sample matches what it would have if they had contacted the whole population - theoretically. The problem is that many surveys don't ask enough people to be able to do this reliably. So in this case, they talked to 1004 respondents. If they are trying to match to 2 genders, let's say 5 age bands, and 3 political preferences (democrat, republican, and other), then you're already at 30 cells. That's 33 respondents per cell IF you assume that each cell has the same proportion - which they most assuredly don't. If they're trying to also match across income or geography, or if any of their cells are much smaller than the others, then there's a much higher risk that at least one of their cells is not going to be representative. So the results may match for gender, if all you look at is gender. Or it might match for age, if all you look at is age. But if you want to know what young women have to say... then there's a much higher risk that the results they've got are skewed.

There are ways around that, but they require either much larger sample size, or significantly more up-front planning for the survey.

Is there anyway a quality poll could be done on the internet?

Yes, it can be done on the internet... but probably not in the way you're thinking. It's usually done via email-ed invitations for participation. The emails are usually provided by panel companies, where people have signed up to participate in market or political research. Then an email is sent to them asking them if they would be willing to participate in an online survey, and a link is provided. Once they follow the link, they enter the respondent ID they were given (so there are no duplicates) and take the survey.

 

[NobleSavage]

Yes, it can be done on the internet... but probably not in the way you're thinking. It's usually done via email-ed invitations for participation. The emails are usually provided by panel companies, where people have signed up to participate in market or political research. Then an email is sent to them asking them if they would be willing to participate in an online survey, and a link is provided. Once they follow the link, they enter the respondent ID they were given (so there are no duplicates) and take the survey.

Do you know the names or urls of any companies that are doing this now?

 

[fromderinside]

Is there anyway a quality poll could be done on the internet?

It's not going to be a representative group. Even if you found someway to avoid people flooding in - like you had a randomized way of finding them instead of them finding you.

Perhaps you could find a representative sample of people who answer internet polls. Maybe you can learn over time how to make adjustments to get a sample that is representative of whatever you're looking for.

If one has a pretty good model of expectations one could design an online poll in accordance with those expectations. For instance the pollster could require signup and confirmation prior to being polled. When criterian numbers of individuals have been polled in a category the poll could be closed to individuals in that group or that individual cold be shunted to another poll which still has slots available.

 

[bongomaniac]

Some of the prejudice against online polling is rooted in the belief that sampling error is the most important source of bias in a survey, and that so long as your sampling frame is broad and representative you don't have to put a lot of effort into weighing the raw results. This might be a defensible position if your sampling frame is the whole population and you invest a lot of resources in trying to get response rates up to a reasonable level; In practice this is almost never done and response rates for telephone polls are now below 10%, so phone pollsters have to put as much effort into weighing their results as panel-based online surveys. The online pollsters are able to accumulate demographic and attitudinal data across a number of surveys, so are better placed to deal with this than phone pollsters. Phone polling can reach people that are missed by online polling, e.g. the very elderly and those with poor literacy, so if the survey is really important you might want to include an element of phone polling to enrich your sample, but the fact is that online polling is cheaper, faster and at least as reliable as phone polling so will continue to grow.