Discoveries

Volume 14, 
#7, 
November, 2016

Tags: 

Why polling fails to predict product development success—and elections
Understanding the zeitgeist of product development (and the electorate)

by 

Sheila Mello and Wayne Mackey

 

Zeitgeist: the spirit of the time; general trend of thought or feeling characteristic of a particular period of time. 

There’s no shortage of opinions about why the polls got things so wrong regarding last week’s election. And there’s no shortage of theories about why technologically wonderful products fail unexpectedly in the marketplace.

We have our own theories, which could help anyone looking to understand what a large, diverse group of people might want and also might help you avoid failure in your own product development approach.

The election highlighted problems with the way we gather and process data about large groups of people—an issue that’s relevant for product developers, innovation in general, and for election polling. Read on for practical lessons you can derive from the failure of the pollsters. And don’t worry: this analysis is strictly non-partisan (no candidate approved this message).

The problem with polling

The problem with pre-election polling was that pollsters

1) called the wrong people and
2) asked the wrong questions.

In other words, they

1) didn’t call a diverse group of people that accurately represented the American electorate and
2) didn’t do the pre-work necessary to understand what questions to ask during polling.

Let’s dive deeper into both.

Identifying and finding the right people to talk to

Many pollsters seem to have lacked the goal of creating a representative set of respondents to ensure that results were valid. To be sure, this isn’t an easy task. Evolving technology (cell phones) means that traditional polling by random-digit dialing to land lines is increasingly difficult, and no reliable system has yet emerged to take its place. That’s one reason why different polls offered such wildly differing projections and why some missed the mark altogether.

Unfortunately, many companies rely on methods of reaching potential customers that are just as flawed. Using customer lists is great if you want to reach people who fit the profile of current purchasers. But what if you want to expand into a new market or create a breakthrough product? Buying or renting a list of names gives you people to talk to—but are those people truly representative of those who might purchase your product? In times of significant change, you’ll need to stretch to find likely new customers and even just to keep your current ones.  

Understanding what questions to ask

It turns out that asking people which candidate they plan to vote for is not an effective way to determine what voters will actually do and thus who will win the election.

It is no more useful to ask customers what features they want in a new product. Instead, in both cases you need to ask questions that help you understand what it’s like to walk in the voter’s or customer’s shoes. What frustrates them? At what point do they sigh with exasperation? When do they jump for joy?

In elections and in product development, knowing what life is like for your target group leads to insights that hide below spoken desires or knee-jerk wants. This is, and has always been, the key to innovating beyond what customers themselves can see and to predicting real voter behavior.

A market-driven approach to election research

Here’s a different approach to election research that might have helped both parties better understand the electorate.

1) Get past the idea that a poll can do the whole job. Instead, determine what problems a diversified group of voters has in achieving their personal goals. Getting the pulse of the populace requires one-on-one interviews to understand what problems people have in getting their jobs done or simply living their lives the way they would like. How do they define success and what are the barriers to that success?

2) Interview a representative—but not necessarily statistically significant—group of people. Segment the total population and match its diversity by talking to three to five people from each segment. This results in images of what it’s like to be a mother, father, student, machine worker, middle-class person, person of color, immigrant—or a member of any other group.

3) Collect an image of the voters and create a set of problem statements. Then determine how satisfied people are with the way those requirements are being met today. Which requirements are delighters and which are must-bes? For example, is fixing the economy a must-be and healthcare a delighter or vice versa? This analysis helps you understand how people will vote based on the candidates’ platforms.

Instead of relying on surveys or focus groups (both of which have serious drawbacks in the world of product development research), a candidate who wants to craft a winning campaign should undertake a market-driven approach to creating his or her platform.

Conduct one-on-one interviews using lots of open-ended questions. Probe for more. Record the interviews. Create an image diagram. Distill the images to the critical few. Define requirement statements that outline what’s missing for voters. Brainstorm solutions. Test them quantitatively.

This would result in a deep understanding of citizens’ pain and perhaps a method to forge policies that would resonate with greater than fifty percent of the population. It would enable pollsters to collect data on the problems people face instead of the product they want to solve those problems. Then, and only then, would the politicians be ready to propose solutions and the pollsters to test those solutions to see if they solve the defined problems.

It may seem counter-intuitive, but if you really want to know how people will vote, in an election or in the marketplace with their cash, stop asking them for the answer and start understanding what will cause them to decide on their answer.

Anyone planning a political campaign is welcome to contact us.

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
4 + 0 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.