No One Picks Up the Phone, but Which Online Polls Are the Answer?

decades past, i used to do market research as part of my occupation. and from what i see now, polling is dead. corporations are shifting from market research to analytics on their manufacturing, distribution, sales and profits. they're not so much interested in what you think as in what you actually watch, hear, and do with your dollars. business is devolving from the enlightenment idea that minds and reason guide our behavior, to just looking at the stimulus and the market response. (and the twitter buzz.) digital media, which can siphon up enormous amounts of tracking and preference data, is more revealing than what you might reply to the questions of a veal pen call center pollster ... with limited language skills. we are all being transformed, incrementally, into herd animals husbanded for corporate profit. polling is too intermediary, too second hand, too speculative. look at the behavior data, and become wise. wise in shaping opinion, guiding opinion, moving opinion. the goal now isn't to find out what you think -- who cares what you think! the goal is to find out how to make you behave a certain way ... a profitable way ... on the path dictated to your future by international corporations and their political lackeys.

Most folks won't answer telephone pollers. Most internet pollers are reaching their respective choirs. Too many polls push. And any moron can do internet stuff--just check social media. It's time to limit polling and to evaluate it for what it is--something with a gigantic noise to signal ratio.

Its very very simple! The only polls that count are the ones that count! That is, only trust polls that, given the results of them, ACTUALLY DETERMINE WHO WINS THE POSITION! In other words, the official ones. For President, this means the Electoral College. Remember ... and and many people I know intentionally lie to telephone and online pollsters. I get really mad when they say "if the election was held today" ... as if that mattered!

Question: "No One Picks Up the Phone, but Which Online Polls Are the Answer?" Answer: None of them as evidenced by 2016 Presidential Election results. Let's go to the videotape... https://www.nytimes.com/interactive/2016/upshot/presidential-polls-forecast.html The Upshot, 08 NOV 2016: "Hillary Clinton has an 85% chance to win. Last updated Tuesday, November 8 at 10:20 PM ET... The Upshot’s elections model suggests that Hillary Clinton is favored to win the presidency, based on the latest state and national polls. A victory by Mr. Trump remains possible: Mrs. Clinton’s chance of losing is about the same as the probability that an N.F.L. kicker misses a 37-yard field goal. For months, we’ve been updating our estimates with each new poll. Today, it’s Election Day, what we’ve all been waiting for, and there will be no more updates. ..." How did The Upshot arrive at their numbers? Polling. In fact, meta-polling. Lesson to be learned: Polls are about as reliable as a Baltimore Colts victory in Super Bowl III.

The polls have become useless. Too many of them. Garbage in, garbage out. Or should I say, you can get any result you want if you tailor the sampling. And then there is who shows up on election day. That can drive the pollsters nuts. Polls nowadays are also suspect because of their timing. Somebody makes a splash in a debate, the next day there's a big news story--and the next day there's a poll. This is what's called a media horse race. The horse that gets left behind is the one carrying the real issues affecting the voters.

Let's get one thing straight, no poll is foolproof. Not a one. I've been polled several times, they always seem to call at dinnertime. To make the calls fun I lie. I tell them what I think they want to hear. Remember all of those polls showing Hillary was a shoe in? Thought so. People use your brain, don't watch the polls don't listen to the pundits tell you about poll numbers and how great they are because they aren't.

Sanders again, pick of and by the people. Biden, the DNC, media, the 1 %. The DNC,the media, the 1% are not going to lose to Trump this time.

Frankly, I am exhausted by the unrelenting and feverish polling as well as the instant commentary. It is misleading and pernicious. The NY TImes prides itself on its running commentary for these so-called debates. If you've ever followed the thread during the debates themselves, this commentary is quite random and hardly ever on point. Sometimes it's simply drivel. Same with the polls themselves. They are meaningless and tend to highlight the flavor of the month. Polling after a so-called "debate," for instance, measures what? This 40-second format is meaningless if you are truly trying to discern candidates' positions. It lends itself to showboating, as we have seen with Kamala Harris's pre-planned attack on Biden. Suddenly, she is outpolling others in the race, despite the fact that we got very little from her on policy, nor do we know the skeletons in her closet. I can no longer watch the MSNBC commentators, whom I used to admire for their intelligence. They have encouraged the hysteria. They are tightly wound people who have an instant opinion on everything. Sometimes, they are quite wrong. Their main pollster has such a "hyper" manner of presentation that I have to mute the TV before he gives me a nervous breakdown. These polls instantly tarnish people who have worked long and hard. CNN reported yesterday that the Sanders candidacy is in BIG trouble. Hyperbole! I plan to keep my own counsel and ignore the pundits and pollsters. They need to calm down.

I can't imagine why anyone would be careless enough to even answer a phone call from pollster, much less to reveal their political preference to someone who might decide to misuse that information. On the bright side, the rise of electoral polls in the 1940's was a net loss for journalism. Readers can still get all the eye-popping headlines they want from the health section, where the stupidest of "studies" are treated like divine revelations.

You mean valid or correctly states the event frequency. Reliability is a property of the consistency of the measure of the event. Most know what you mean.

While I will not argue it is perfect, 538.com, which used to be housed by the NY Times, does provide an evaluation of different polls along with how they do that evaluation. https://projects.fivethirtyeight.com/pollster-ratings/

The website FiveThirtyEight rates non-online pollsters: https://projects.fivethirtyeight.com/pollster-ratings/?ex_cid=irpromo. Most recent update was May 30, 2018. Rasmussen is probably overrated in view of its 2019 polls far out of line in favor of Republicans compared to all other polls.

Please don’t encourage anyone to call our home for any reason. If we want to talk with someone we will call them. Thank you!

I’m wondering if the attention given political polls actually influences the vote (the tail wagging the dog): an undecided voter may choose to vote for a candidate based on their standing in the polls — everyone likes a winner.

1."Is the U.S. on the right track"? Yes or No How do you define "right track"? 2."Do you support or oppose the Iran Nuclear Deal?" Yes or No? Can you summarize the details of the Iran Nuclear Deal and if you cannot, how can you answer? In general, many poll questions are so simplified and superficial that the results are meaningless

Robocalling has been the death knell of telephone polling. Even if caller ID shows a polling call, the recipient has no way to know if it is genuine.

Monmouth, Marist, Quinnipiac are my favorites. They are ranked "A" by FiveThirtyEight based on many variables. And they're pretty neutral when it comes to polling Democrats and Republicans equally. Then there's Rasmussen, which usually gets a "C" and polls too heavily Republican. So when you get averages, like on RealClearPoltics, those averages can't really be trusted because they're being skewed by heavily partisan pollsters.

While the article isn’t clear about the percentage of “live interview” (telephone) polling is done by calling landlines or cellphones, I assume both. But I’m 61 and my husband is 65 and we don’t have a landline, nor do most of our same age friends. And as “old” as we may be, we’ve actually managed to figure out how to block “anonymous/private/unknown” incoming calls made to our cellphones which begs the question, who IS answering the phone? I would guess that calls to landlines during the day (and early evening) are answered by elderly people probably living in rural areas where cell service is “spotty” if even available. That alone would absolutely skew the results.

Anyone with even a passing knowledge of statistics and surveys knows that online polls are inherently unreliable because of the impossibility of establishing a representative sample.

Polls are only relevant if they inform a particular action. They make sense for campaigns to adjust strategy, and they make sense for news sources that want clicks about the latest poll. For the rest of us, the only poll that matters is the election, as none of the others actually matter in terms of making decisions.

Well done! Polling is considered sort of scared and this helps to demystify it. An added problem is that the questionnaire design is often terrible. I participate in surveys when possible and find that from leading/biased wording to incoherent question order the poll leaves me wondering how trustworthy the results are to a campaign. When I worked in market research we constantly fought the battle of "bad data is better than no data". I suspect in a political campaign it is even harder.

The number of people surveyed is the first item I check when I see poll results presented graphically, and I automatically ignore results based on input from only a few thousand people. No one can convince me that results based on such small samples are significant. I also discount any online polling due to the issues identified by Mr. Cohn. More bad data creates more bad results. I would rather see the candidates' report of contributions, especially the number of individual contributors. After all, those statistics better reflect the old adage of putting your money where your mouth is.

I don't trust any of the polls anymore. The ones conducted by phone only reach land phones which are disappearing quickly and represent a smaller and smaller segment of older voters. And after watching how biased online sites negatively impacted the last presidential election, I wouldn't trust their results anymore than I could throw them.

The sad fact is that all polling appears to project a liberal bias, creating a false sense of security for liberal candidates and causes - anywhere in the world - and might even be responsible for some liberal voters to actually stay home on Election Day - the only poll that really counts - believing that "we have this in the bag," and not to worry. As evidence for this premise, I submit the 2014 Colombian FARC treaty, Netanyahu's reelection, Brexit, the USA election of Donald Trump, the Australian 2019 election of Scott Morrison, and any number of European national elections since 2015 that belied expectations and handed victories to conservative candidates. As explanation for these universally skewed polls, I can only offer the notion that poll respondents don't want to appear conservative among a roomful or houseful of liberal friends. And as a word of advice to all liberal voters: Get to the polls, regardless of how the numbers appear in the days before an election.

I used to work in polling, a coupla places for political polling, others for market research. My impression was there were enough respondents on the phone who are only responding to have someone to talk to or to merely humor/prank the pollster (yes this happens) that any outcome was sufficiently skewed past the known skewing variables to render results moot. Sometimes there were clients who fudged the numbers to work in the favor of some desired perspective; ergo the polling firm provided the evidence in favor of that perspective. I hear people mentioning the whole Hillary v Trump thing from 2016 and how the polls were wrong, but actually they weren't: Hill'ry won the popular vote, vindicating the polls. It's just within those three narrow delegate theaters that Trump was able to take the greater count of delegates.

At this point, I would not trust any caller to, in truth, be what he/she represents themselves to be. I feel privacy laws in our country have now been fully tailored to corporate interests, so no, I'd never knowingly share any data or intentions over the phone with anyone -- the 1950's are not coming back. The logical endpoint for pollsters will intimately be for them to mine the trove of inferential data that commercial concerns have gathered about me, and project from there. One could make a credible argument that *that* data (with proper psycho-graphic processing) more accurately reflects what I will actually do/vote for, etc than what I *tell* you I currently plan to do.

Caller id resulted in a big change for polls. A second problem is who actually answers on line polls. Research suggests considerable bias. Third, about 10% of the US still don't have internet access. Fourth, and more important to all polls is the questions. Bias and ambiguity in the questions is rampant in many polls that is intended to get a specific result.

Considering what happened in 2016 and the fact that presidential elections are won and lost in the Electoral College, I'm interested only in state polls- and specifically those that focus on Pennsylvania, Michigan, Wisconsin, Ohio and Florida. As things now stand the rest of us scarcely matter.

Great points to ponder! As a professional market researcher (and an old one), researchers have struggled with representative sampling in their polls for as long as social scientists began polling. Also, as many readers are aware, polls are just a snap shot in time. In the age of diminishing attention spans this becomes even more problematic when dealing with online polls. Peoples opinions can flip instantly with the next twitter post. Political polling is the most difficult to ascertain objectively. The wording of questions could bias answers (intentionally or unintentionally). The nature of polling itself can be biased. If people are wary of strangers/gov't or other conspiracy beliefs, the tend not to participate or purposely respond untruthfully. Either way, data may reveal one thing and reality another. Behavioral data sets may be the best predictor of future polling (yes, big data set analysis is being done on everyone of us every day).

As a marketing researcher with 40 years experience in the business, let me tell you just a couple of problems with online polls. First, weighting by demographics is a farce. It’s only going to work if there is a 100% correlation with attitudes. It’s attitudes we are measuring. Do old people always vote for one party? Of course not. Put another way, if you know someone’s age can you predict if they will vote for one party or another? Of course not. Second, you can’t cite the amount of sampling error with non probability samples and if a firm does it’s a sure sign they are not reputable and know nothing about the science of statistical probability and sampling. Third, the best way to do polling is to do continuous surveys every week or every month so you can develop a trend line. I instituted this approach in Canada and borrowed from an extremely successful firm doing it in the UK. Slices in time polls are misleading. Period. Because the sample is affected by what’s happening in the media or on TV that week, that day, announcements by politicians etc. Only by doing continuous polling do you get a meaningful result. But you can’t keep asking an online panel the same question because it affects their answers (although you could make tranches of the panel). So it’s hard to do it with online panels. In sum it’s obvious why online surveys are inaccurate.

Whether online or over the phone, nothing will stop people from telling pollsters what they think they want to hear. In this openly hostile, emotionally charged political environment, there will be many and growing who will spout the party line. The so-called Bradley effect. That's not in any pollsters control no matter how you ask the questions.

All fair points, Nate, but what is worse: An individual poll with some potential error, or a non-transparent "predictive model" built atop hundreds of polls that magnifies the individual error a thousand fold? If polling and data journalism were the 2008 Financial Crisis individual polls would be bad mortgages, and Nate Cohen's (and Nate Silver's) models would be the CDOs built on them.

I am part of the problem. For one thing, it’s too hard to know today whom you are really talking to. For another, the yes/no or choose-one formats of the questions leave me feeling boxed in so tightly that I wind up expressing opinions that don’t accurately reflect my feelings. Like many others, I’ve quit automatically answering my phone. I have also become very skeptical of even reputable polls, knowing how important the framing of the questions can be. Still, I confess I do look at the final results with interest.

The bottom line imo is that modern polling methods are like democracy and marriage, flawed institutions but nobody has yet come up with anything better.

Polls are unreliable and unscientific. To be statistically reliable, the sample power size should be high. We now ignore phone calls or online surveys and the number of those who participate in polls is dismal. How many times have these polls been proven wrong in recent elections? Why do candidates, voters, and the media rely on this "fake" indicator of performance of candidates? Using these polls to apprise people of which candidate is doing well is a farce. Look into the candidates' values, policies, sincerity, intelligence, qualifications, and their ability to pull through a crisis. We long for a Dem candidate who can stand up against Trump and his grandiose and lawless ideas. Buttitieg has all these qualities. He is calm, sincere and thinks through what he says. What comes out is a thoughtful response that is spot on. He does not raise his voice or interrupt others to get his message across. People listen when he speaks. He commands respect and authority. Imagine Trump bullying him during the debates. He will respond in a cogent and concise manner that Trump will not know what hit him. Everyone in the candidate pool who will be bullied by Trump will get rattled, except for President Buttitieg. His ideas are innovative and in tune with the times. He gives us hope that we can regain our smeared country from Trump and his cronies. Forget about polls! Look and listen to each candidate, especially Peter Paul Montgomery Buttitieg.

I generally don't answer my landline phone. I let my machine do it. You can thank the telemarketers for that (Yes, I'm on the "Do not call" list). A half dozen frivolous calls a day from telemarketing computers will do that to a person (Two calls since starting this comment). So I've never participated in a telephone poll. Maybe if we could gain back control over our telephones, telephone polls would be more effective. I'd love to wax eloquent over the telephone on politics to some pollster.

My friends and I never answer our phones unless we know the caller ID. So we don't participate in any polls. However, I am a member of a popular poll service online and I think the questions polls, in general, ask are too narrow. During WWII, in England, there was a group that actually talked to people every day and reported their findings to Churchill. He had intimate knowledge of what his people were concerned about and what they wanted. We have to figure out a compromise in our polling. We really have to allow people to "comment", and we have to ask more than 5 questions. We also have to try to qualify answers. Often I answer "yes", but under certain circumstances the answer would be "no". Don't they care to know? Wouldn't that be a big reveal? I see a repeat of 2016 and here is how I know. Look at the Trump rallies. Stop "not covering" them (CNN & MSNBC). And look at the Dem rallies. It's right there in front of us!!

I have been calling and walking for candidates for years. The only answer is big samples using calling, internet, mailing and walking all together. People haven't been answering for many years. Maybe if you don't get a response you keep trying until you do. In 2018 election no one answered their phones.

We don't answer the phone if a name and number comes up that we don't recognize. If the caller is someone we would want talk to, they would leave a message.

Thank you so much for this very informative article!

The stupid media and our corrupt politicians love polls but we do not

Where are the professional survey organizations in all of this? Why isn't there a professional standard that says "non-probability and low-response samples of the general population are not to be trusted." And perhaps a warning label that says "What you are about to read is garbage."

This headline is misleading. Typical live answer rates are about 25% on a single dial through on a voter list. It can be as high as 40% if the weather is poor. Live call centers can reach about 75% of the voters if they dial the list multiple times over a few days. The problem is that 40%-50% may refuse to answer the questions. I don't see how online or text polls will solve that problem.

This article omits an important question. When polls that do not randomly sample the population claim that the sampling error is plus or minus 5%, how do they know? In the absence of random sampling, there is no reliable way to estimate error. By "reliable" statisticians mean "works consistently." It is true that, after an election, one can see how close various polls came. But of course, that's not what polls are for.

This headline is misleading. Typical live answer rates are about 25% on a single dial through on a voter list. It can be as high as 40% if the weather is poor. Live call centers can reach about 75% of the voters if they dial the list multiple times over a few days. The problem is that 40%-50% may refuse to answer the questions. I don't see how online or text polls will solve that problem.

Polls are a form of social control. I have often observed that by only allowing yes or no answers, you are forced to accept the framing of the question which may be absurd to you. There was a time in the last election when the phone polling was incessant. I resent the constant interruptions and I often answer phone polls with the opposite of what I actually think in the hope that it may help push someone I am against to become overconfident.

I vividly remember the little bar graph the Times had on the front page (online) during the runup to the 2016 election. It was a 'sliding scale' showing the relative favorability between trump and Hillary It kept showing Hillary winning by comfortable margins, like 80-90%. Polls? Hah. The ONLY poll worth noting is the one tallied after Election Day. Anything and everything else is pure speculation- weighted, of course.

I'm one of those people that has declining confidence in election polls yet I can't stop checking them pretty much every day. I'm sure I'm not alone. What should we name that phenomenon?

I have seen polling questions on Facebook, but I don't participate. Why not? Because there's no anonymity, and I don't want to get in trouble for an unpopular answer. That's assuming that a Facebook poll is legitimate in the first place.

What makes this phenomenon uniquely troublesome in the US is that as the older generation is the one that answers the phone calls, but is dwindling, it's ALSO the case that the US is SO skewed against young, working people voting that the phone samples were still valid, far longer than they should be. But they can't be relied upon, it's an absurd technique already, and soon to be archaic. Regardless, the online techniques are the only ones worth the effort now, because they have a future.

It hurts to say this, but, Good Riddance! We’ve gotten to a place where politicians spend large amounts of time soliciting major donations, and relying on polls for alarm bells warning against the most egregious donor demands. Ending polling would force them to spend more time with us, the true constituency. The same applies to corporate media, who endlessly pass on poll results to avoid deep research on the most controversial policy moved. Ditto for policy moves that ought to be controversial. An alternative would be well-chosen focus groups, as sometimes appear on public television and a few other purveyors of real news. Here people ate given time to consider and explain their answers to specific questions or events. Usually the group consists of cultural and economic mixes our society to seldom affords us in public spaces. The answers often surprise my stereotyped expectations, and usual goad more careful analysis on my part. Polls do none of these things. They either anger or confirm, probably playing a role in today’s polarized society. Every now and then, I do participate in a telephone survey. It has to be clearly local, and ask nuanced questions. If it gets too black-and-white, I terminate. Does that help anything? Who knows? But at least I have refused to play the values-confirmation game one more time.

I never answer the phone anymore due to robocalls. Maybe if there's a caller id. I've stopped automatically dismissing requests for online surveys. Maybe they'll become more normal.

Why would anyone tell an unknown person on a phone for whom he would vote? Polling does not make much sense face to face, but via phone? Calibration would to account for fact that responder is not normal.

Thanks for this fascinating article. It would be nice if online polls were so labeled. Especially if some have such a high rate of margin of error. Since many important decisions are being made by polling results ( ex. the Democratic debates inclusion policies) and so much media activity is driven by polls, educational articles like this one are more valuable than ever. There is so much spin and in some cases outright lying in politics that polling needs to be put in its place. In addition, what about taking into account the very large segment of the population that refuses to participate in polling? This skews the results, as well.

Are polls necessary? How might politics change without them? What if politicians did and said what they thought was right and let the consequences play out without worrying about poll numbers? Would journalists have to discuss the nuances of politicians' positions on the issues instead of the polls? But don't worry: I'm sure that soon it will be more accurate to make predictions and take the public's temperature by gathering massive data from social media and online activity than by asking people what they think anyway.