Why You Should Not Trust Polls

During my 30 years of consulting I designed, administered, and analyzed a variety of employee and customer surveys (polls).

 Surveying is an important and efficient method for collecting, organizing, and analyzing the input of hundreds and perhaps thousands of people. A well-designed survey will include demographic identifiers that can help you slice and dice the data. The Internet can capture, update, and report trends and findings in matter of seconds. 

 The quality of the survey results is impacted by a variety of snafus, some technical, others administrative. The most common and generic mistakes people make are:

·      Sampling.  The sample must mirror the population you are surveying. As a federal republic, we elect presidents on a state-by-state basis through the Electoral College, and not on a national referendum basis. That is why the total number of votes is trumped, no pun intended, by the cumulative results of the states. Most polls rely on fix-line phone calls. We know that millions of people have opted for the cellular route. There is no comprehensive phone listing for cell numbers. Therefore, it is easy to miss the input of folks who have chosen to abandon fix lines.

·      Wording. Questions need to be clear, concise, and free of desirability and double binding (two or questions in one).  This is a common problem. The words used must make sense to the respondent. Asking people to pass judgments on topics for which there is an expected (socially desirable) response leads to getting the answer you are fishing for. We saw this in the last election. Folks who supported Trump were not always willing to say so, afraid perhaps of social consequences. Asking questions with an “and” in the middle makes it tough to distinguish which of the two parts is the respondent answering. Asking folks to comment on a topic they know little about is also to be avoided.

·      Snapshots.  Surveys are snapshots in time, not the whole movie. When sequencing the snapshots, you might capture a trend, but not the final movie. There are events (October surprises, for example) that can alter the trend’s trajectory abruptly. We saw this phenomenon in action in last presidential election. On the 27^th of October Clinton was leading; on November 8^th Trump won. In the interim a variety of events took place that might have shifted enough people to the other side.

Defense Mechanisms

When the results of a survey (poll) are “hot” or “shocking”, you can anticipate one of three types of response:

·      Attack. People will criticize the questions, how they were framed or asked, or the context in which they were asked. Some will attack the credibility or the qualifications of the persons conducting the survey. Others will raise questions about the validity and/or reliability of the results. This response is typical of red-hot personalities.

·      Rationalization. People will intellectualize the findings, provide reasons for the results, and engage in what if explorations. Some will debate the statistics. It is an attempt to cool the results down in order to avoid its “hot” or controversial impact. This response is typical of cool-green personalities.

·      Whatever. A few will choose to go with the flow invoking the classical California defense mechanism of “whatever.” It is a nice way to avoid dealing with the controversy or hot potato. This response is typical of the “yellow or sunshine” personality.

Defense mechanisms are responses to a perceived threat. Unless data are cooled down, it will be hard to proceed to the “so-what” and problem-solving phase. We saw these factors in play after the 2016 presidential results. Polls predicted a certain outcome when in-fact another came out.

Mid-Term Elections

We will soon be voting for the entire House of Representatives and 1/3 of the Senate. Every week or so we are presented with poll results.  It is hard to make sense of what they mean. Some experts tell us that it will be a vote for or against Trump. Others will remind you that they will reflect the local situation.

Historically, people vote their pocketbook. The proverbial question is: Are you better off now than you where two years ago? Many will answer this question through their self-interest lens.  Am I earning more? Is my stock portfolio doing better? Am I more secure in my job? Are there more and better job opportunities? And so on.  Others will answer based on their philosophical bent. Do I like Trump’s style and personality? Are we going in the right direction? Are our values being tarnished? And so on.

It is hard to predict which side will carry more weight: economics or personal values. Most folks do not like to change horses in the proverbial mid-stream. If they perceive the country moving in the right direction, they will vote for the status quo. If they perceive that they are worse off now than they were two years ago, they will vote for the opposition party. It will be hard to show that people are not as well off economically given the passage of the tax bill, a surging economy, and a historically low unemployment rate.

The other variable is the message. Is the message uplifting and one that speaks to the voter’s concerns and needs, or is it a message concocted by party extremists pushing an agenda with which the voter does not relate? You cannot be against everything. You have to be for something.

Interesting footnote. The Democrats who upset Republicans in Alabama, Pennsylvania, and other states in the past 2-3 months interim elections look and sound more Republican that the Republicans they replaced.

We are a divided country. Red in the middle, and Blue along the West Coast and the North East. Excesses on both sides of the aisle will surely affect the final result. We are a long way from October and possible surprises.  However, we can predict that one side will rejoice, and the other will cry foul.