How do you plan, design, and conduct a conjoint analysis survey?

2019/09/23

by Thomas Vladeck

The major steps are the following:

  • Build the attribute map
  • Design the experiment
  • Field the survey
  • Run the regression
  • Synthesize the results

Build the attribute map

Conjoint experiments entail having the survey respondent choose between 2+ hypothetical options (or, less commonly, to rate a single choice). The options consist of attributes which have different levels. The attributes are dimensions and levels are the values they can take. For example, if you’re doing a conjoint on car preferences, transmission might be an attribute where as automatic would be a level. The most important attribute in most experiments is the price, with different price points as the levels; others include the brand, form factor, shape, color, etc.

 

Ideally your attributes are independent of each other so you don’t have to worry about prohibited combinations, but often some interaction is inevitable. As a very rough rule of thumb, you want somewhere between 3–6 attributes each with somewhere between 3–6 levels.

You also have to choose how many choices you will show in each “choice task” (typically one survey screen), and whether there will be a “none” option.

Design the experiment

Which combinations of features you show for each concept on each screen to each respondent can increase or decrease the amount of statistical power (how much you learn from the experiment).

Randomized experiments (where the combinations are chosen at random) typically do pretty well, but if you need to be as efficient as possible, you need to use specialized software, like the AlgDesign package in R, or Sawtooth software.

Field the survey

To field the survey, you will first need to program it. If you’re willing to get your hands dirty, this is doable in Qualtrics (we do it all the time), but if you need something done for you, Sawtooth is a standard platform for this.

To get respondents, you can use mTurk on the cheap / low-quality data end, quota panel providers like Dynata in the middle, quasi-probability panels like YouGov on the upper end and then very high quality probability panels like Amerispeak or Knowledge Panel on the very high end.

Run the regression

The standard model for conjoint is the multinomial logit model, which explains the choices that respondents made by assigning each level a “utility” or “part-worth”. The relative “utilities” of the choices presented to the respondent (after getting plugged into a special equation) output a probability for each choice, and the goal of the model is to change the utilities such that the probabilities of the observed choices are as high as possible, and the probabilities of the choices not made are as low as possible. Often the model assumes that each respondent has their own set of utilities (the hierarchical multinomial logit model).

To run this model, you need to be very handy with statistical modeling and/or to use, again, Sawtooth.

Synthesize the results

There are several ways to unpack the results of the model, but they all come down to different ways to understand the tradeoffs between the levels within an attribute.

The most common way is to compare the change in utilities with a change in price. “Changing from an automatic to a manual has the same impact on choice as increasing the price 14%”.

Another very common way is to build a simulator that estimates the “market share” for a set of concepts (often including concepts that actually exist in the market).

conjoint choice analysis

Thomas Vladeck

Written by Thomas Vladeck

Tom was inspired to start Gradient by the cutting-edge market research performed by his advisors at Wharton, where he received his MBA in marketing and statistics. Prior to Wharton, Tom received a master’s degree from the London School of Economics and studied math at Pomona College. In a prior life, Tom produced quantitative models for global climate policy reports.

Comments