An extremely confusing survey question appears in the November edition of Harvard Business Review. The authors could have asked a simple and straightforward version of the question. Alas, that would have got in the way of creating an overly elaborate diagnostic “tool”. And goodness knows that we need more of those.

The Peripheral Vision Scoring Tool is a self-diagnostic questionnaire designed by Day & Schoemaker (2005). By filling out the questionnaire and tabulating the answers, you are provided with summary statistics indicating your vulnerability to emerging business threats, especially those coming from unexpected sources. Many business leaders are not “asking the right questions” about their capacity to “scan the periphery”, say the authors. And by “right questions”, Day & Schoemaker are apparently referring to ones like this (figure 1).

Respondents are asked to tell of the number of surprises they have experienced on a scale of one to seven. Now, set aside the ambiguity of what it means to be surprised by “high-impact” events. How do you answer this question given the discordance between (a.) the numbers on the scale and (b.) the labels on both poles of the scale?

Consider an example. If you had been surprised twice by high-impact events during the period, then what number on the scale would you choose? The answer is: there is no response which makes much sense.

We know that “1″ corresponds to no surprises in the past three years. So if you had not been surprised, then you would know which option to select. Had the other end of the scale had been labeled “Three”, then a single surprise would be option “3″, two surprises would be “5″ and three surprises would be “7″ (and “2″, “4″ and “6″ would represent an additional half-surprise, respectively, if that makes any sense). There is a sketchy logic to this but it is confusing, to put it mildly. The problem is that you wouldn’t be able to answer if you had, for example, four surprises. Thus, the authors labeled the pole as “Three or more” for the sake of exhaustiveness. A “7″ could mean three or it could mean three hundred. This makes the scale quite silly as a gauge of magnitude. In any case, the respondent is left with only the vaguest of clues as to what options 2 thru 6 refer to. Answering the question requires far too much mental gymnastics.

Five- and seven-point scales have been a mainstay of social scientific inquiry since Rensis Likert popularized them in the 1930s. Ideally, the scales include some descriptor to tell what each option on the scale means (”strongly agree”, “agree”, et cetera). Even if every option on the scale isn’t labeled, one should be able to infer using the options which are labeled (usually the poles and the middle option). If a scale is very large (nine-points and above), then the respondent should have some notional grasp of the degree being conveyed. Unfortunately, use of Likert scales has become so widespread and a few reckless researchers have lost sight of these basic principles.

I can imagine why this mistake was made. There are 43 other questions in the questionnaire, all of which use seven-point Likert scales. Day & Schoemaker probably opted to ask all questions in a consistent way to make it easy to add up the answers into summary scores. Otherwise, the calculation would involve a complicated formula. Sadly, it is not uncommon for researchers to shoe-horn a question into an inappropriate format so that they can perform a particular type of analysis on the results. This also happens when a client badgers a researcher to drop a pet question into an otherwise coherent battery of scale questions.

As an aside, this article is a sneak preview of a forthcoming book called Peripheral Vision: Seven Steps to Seeing Business Opportunities Sooner. In my experience, whenever a management guru claims to have a seven-step solution to a problem, it usually takes seven-or-more steps.

By Peter Stoyko

REFERENCES