How do you come up with a number for the ‘value’ of something that can be meaningfully compared with alternatives. There are many approaches. Ulwick (What Customers Want) discusses this when he derives his ‘Opportunity Score’ (Importance + Max(Importance – Satisfaction, 0) = Opportunity).

In his book, Ulwick states ‘Research methods that focus on solutions often require the use of complex scaling methods (such as paired comparisons, repeat-pairs techniques, balanced scales, comparative scales, constant-sum scales, graphic-rating scales, and rank-order scales) because they are trying to quantify something extremely subjective – what solutions customers like best.’ He goes on to state ‘Critics often argue that one can’t subtract satisfaction from importance, that by doing so is like subtracting apples from oranges. … Their argument may be true when it comes to evaluating solutions, but it does not hold up when talking about jobs, outcomes and constraints.’

Ulwick is on to something here, but his explanation is incomplete and unsatisfying. Ulwick himself is trying to measure subjective qualities (a complaint he levels at these other methods) so what makes his approach different? Also, what is the essential difference between measuring ‘solutions’ and measuring ‘outcomes’ that make his final number any more meaningful and useful that numbers derived from the ‘complex scaling methods’?

The essential difference between trying to measure a person’s response to a ‘solution’ and measuring their response to an ‘outcome’ is one of context. The outcome measure is ‘context free’, that is it can be judged by a person independently of any other item and this makes it 1) much easier for a person to state an opinion as a number between 1 and 10 and 2) makes the number given much more consistent between the various outcomes being measured.

An example from Ulwick's book illustrates this:

• Solution question - How important is self leveling?

• Outcome question – How important is it to minimize the time it takes to level?

The first question requires the person to understand the feature of ‘self leveling’ and then to mentally rank it with respect to other potential features, both those that have been mentioned in past questions and those that may be asked about in the future. The term ‘Self leveling’ has a context.

The outcome question, on the other hand, is free and independent of any context. It doesn’t require understanding what self-leveling is and doesn’t require comparison to any other outcome.

This then is the essential difference between the two approaches and what allows Ulwick to subtract satisfaction from importance and get a number metric that is robust.