Best Tips for Sound MaxDiff Design

Introduction 

In an increasingly competitive landscape, the maximum difference analysis (MaxDiff) has emerged as a vital technique for identifying the key drivers of consumer preferences. When thoughtfully designed, MaxDiff serves as a reliable compass to guide brands toward impactful messaging, product enhancements, and customer experiences. In this post, we’ll provide best practices for constructing an effective MaxDiff experiment to yield the nuanced, actionable insights that will ensure your compass points north. If you’re still not sure a MaxDiff is the right discrete choice experiment for you, you can navigate to this blog post to assess if it’s the method that best maps to your needs.  

The direction-revealing power of MaxDiff comes from forcing respondents to make tradeoffs between product or brand attributes, rather than simply rating them individually. This elicits more nuanced insights into the relative importance of each attribute and how they influence consumer choice and intent in a setting that more closely mimics the real world (globe). With the proper design considerations described in this post, MaxDiff can provide clear direction to brands on where to focus innovation and messaging for maximum impact. 

To arrive at the full potential of MaxDiff’s ultimate destination, it is critical to invest time defining clear research objectives, selecting impactful attributes, optimizing display, and crafting easy-to-understand instructions. Adhering to these best practices will enable brands to leverage MaxDiff as a north star, guiding strategic marketing decisions and improved customer experiences. For those looking to get acquainted with MaxDiffs before learning best practices, journey toward our helpful primer. 

Defining the Research 

Before embarking on the journey of fielding a MaxDiff to customers, it's critical to first define the destination by setting clear goals and response frameworks. This necessary groundwork provides direction for downstream design choices. Taking the time upfront to clarify the research objectives, define the appropriate anchors, and craft the question wording sets the stage for effective MaxDiff design. Clearly articulating the goals of the research enables alignment between the study design and the desired business decisions it will inform. Selecting appropriate anchors frames how respondents view the tradeoff tasks while clear instructions ensure they understand the exercise.

Clearly State Research Objectives 

Taking time to identify focused research objectives is the crucial first step of MaxDiff design and the true north that guides all downstream decisions. Be specific about the core questions you want the analysis to answer, since this will inform the optimal anchors and wording to use. For example, if your objective is to understand key drivers of purchase intent, "most important" and "least important" anchors make sense. But if you want to uncover basic requirements vs. excitement factors, "must have" and "nice to have" work better. Avoid broad objectives—well-defined goals lead to concrete recommendations. 

Document your objectives and refer back to them as you make choices about anchors and wording. The anchors should match the objectives to appropriately frame the tradeoff exercise. The wording must clearly instruct respondents to make tradeoffs in a way that aligns with the objectives. Continuously refer back to the research goals as you finalize the anchors and wording to ensure alignment. 

Select Appropriate Anchors 

Anchors create the lens for the tradeoff exercise. The anchors should directly hook to the defined research objectives. For example:

1. If exploring key drivers of purchase: "Most Preferred" and "Least Preferred" 
2. If interested in performance: "Best" and "Worst" 
3. If determining advocacy: "Recommended" and "Not Recommended" 
4. If assessing impact: "Effective" and "Ineffective"
5. If distinguishing product features: "Premium" and "Basic" 

The possible anchors that can be used are endless, making the MaxDiff a highly flexible experimental design. However, while the number of anchors that can be used is high, it is important that the two anchors you choose are distinct (and ideally opposites) from one another. When in doubt, carefully consider which two anchors best match the research objectives and attributes being tested (more on this below!). 

If you’re unsure which anchors will lead to the best results, test two or three options with a small representative sample. By modeling these early results, you can determine which framing yields the clearest insights based on respondent feedback and result stability. The anchors influence how respondents view the attributes, so it's crucial to evaluate options to find the optimal pairing that provides the desired insights.

Craft Clear Question Wording

The question wording for MaxDiffs is paramount to gaining the insights you’re looking for. Here are some tips on crafting the wording for a MaxDiff:

1. Use simple, unambiguous language to explain the MaxDiff exercise. 
2. Consistently incorporate the selected anchors to frame the tradeoffs. 
3. Avoid complex verbiage and industry jargon. Opt for clarity and simplicity in the wording to ensure easy comprehension. For example, “Of these features, please select the one that is most important to you and the one that is least important to you.” 

When in doubt, pre-test two or three variants of the question with a small representative sample to identify the optimal wording based on respondent feedback and correctly completed tasks. 

Designing the MaxDiff

With clear direction set, the next step is to thoughtfully design the attributes at the heart of the analysis. Careful selection and organization of attributes ensures the MaxDiff exercise yields actionable insights. Choose attributes that map closely to the research objectives. Determine the optimal number to include. Vary attribute subsets and ordering to minimize potential biases.

Choose Impactful Attributes 

Attributes make or break the insights that can be gleaned from a MaxDiff, so carefully select ones that map closely to the defined research goals. In a sense, the attributes selected act as your hypotheses for the experiment. Each attribute you choose should hypothetically be a driver of your outcome variable, which in the context of a MaxDiff are your anchors (e.g., preferences, recommendations, product feature importance).

Here’s a checklist of what to think about when choosing attributes:

1. Be sure to cover the key decision factors and topics of interest. For example, if exploring drivers of purchase intent, include a mix of potential differentiators like innovative features along with basic requirements like price. 
2. Include a strategic blend of exciting differentiators as well as basic must-have factors to enable nuanced findings. 
3. Limit the attributes to the most impactful ones to avoid overwhelming respondents with too many tradeoff tasks. Aim for 10-15 attributes when starting out.
4. Brainstorm a broad list of potential attributes, then narrow those down based on importance to objectives, ability to differentiate, and variety to represent key decision factors. 
5. Confirm attributes with customer interviews or surveys to ensure relevance. 
6. Revisit selected attributes once analysis is complete—less differentiating attributes may need to be replaced in subsequent studies.
   
   

Determine Optimal Number of Attributes

When selecting attributes, aim for balance. Typically 10-20 attributes maximize meaningful insights while minimizing respondent fatigue. Too few can limit the actionability and depth of the results. On the other hand, testing too many attributes increases dropout rates and introduces cognitive strain. 

For those new to MaxDiff, start on the lower end, around ten attributes, for initial surveys. Gradually expand the number in follow-up studies once you determine respondents can comfortably complete more choice tasks without fatigue setting in. To determine the appropriate number of attributes, start with a small pilot study using varying attribute amounts. Assess respondent feedback, completion rates, and result stability. The optimal count provides differentiation without overburdening respondents. 

Optimally Display the MaxDiff

With strategic attribute selection complete, the next step is optimizing how the attributes are displayed through subset variation and random ordering. Present different subsets of the attributes across the MaxDiff questions to minimize biases respondents may develop based on the competitive context. Rotating the choice sets provides a more holistic view of preferences less skewed by the influence of other attributes. 

It’s also important to only show a subset of the attributes in each task. Aim for five to seven questions with different attribute subsets. Any more generally provides diminishing returns as it is harder for respondents to make a tradeoff. 

Choose a Reasonable Number of Tasks

Lastly, it is important to choose a reasonable number of choice tasks. Too few and you may not have enough data to obtain reliable results; too many runs the risk of burning out the respondent and getting low quality responses. Select an appropriate number of MaxDiff choice tasks per respondent to maximize reliable insights while minimizing fatigue. In general, more tasks provide diminishing returns as respondents become weary. Start with 10-12 tasks for ten attributes and reassess based on data quality. 

For those who are analytically inclined, statistical simulations can be leveraged to determine the minimum required for acceptable reliability. For those looking for an easier-to-use calculator, Sawtooth Software offers a point and click interface to determine the minimum number of choice tasks needed to have an acceptable margin of error. All you need to know for this calculator is the total number of attributes and the number of attributes you plan to display per set.

Pilot testing can also be used to determine the optimal number of choice tasks. Begin by launching several surveys that vary in the number of choice tasks. Once you have the final data from each survey, assess completion rates, respondent feedback, and attribute score consistency. 

Lastly, more tasks may be required if you plan on analyzing subgroup differences. For more complex MaxDiffs it’s best to consult experiments who have the statistical background to calculate everything we’ve discussed thus far.

Conclusion 

Well-designed MaxDiffs serve as a reliable compass to reveal meaningful insights to guide strategic marketing decisions. Taking the time up front to clearly define research goals and appropriately frame the task sets you on course for high quality MaxDiff data. Carefully curating a bindle of impactful attributes at a manageable number enables actionable insights. Optimizing display and pre-testing identifies any dangerous waters before launching the full survey. With thoughtful design and preparation, MaxDiffs have the power to guide business decisions and maximize your competitive advantage. 

Do you need support navigating the potential of MaxDiff analysis? Let us know. We have extensive experience designing and analyzing MaxDiff surveys to uncover the key factors driving consumer preferences and choices. Our team can help you leverage MaxDiff as a strategic tool to inform product development, marketing campaigns, and customer experience enhancements.