How to Resolve Conflicting Customer Interviews

When customer interviews yield contradictory feedback, the solution is never a majority vote. Instead, you must resolve the disagreement by mapping the conflicting data against user context, behavioral evidence, and potential interviewer bias. By using an evidence matrix, early-stage founders can determine whether conflicting answers represent a flawed interview process, a misunderstanding of the problem, or simply two entirely different customer segments.

Here is a step-by-step guide to analyzing conflicting customer feedback and deciding what to believe.

Why Customer Feedback Contradicts Itself

Before throwing out your research, it is critical to understand why users give conflicting answers to the same questions:

• Mixed Customer Segments: You may be interviewing two distinct personas who experience the problem differently. What is a critical pain point for a mid-market manager might be irrelevant to a freelance consultant.
• Aspirational vs. Actual Behavior: Users often report what they wish they did rather than what they actually do. One user might describe their ideal workflow, while another candidly admits to using a messy spreadsheet.
• Interviewer Bias: Leading questions, changes in your tone, or introducing your solution too early can cause users to tell you what they think you want to hear.

Step 1: Separate the Segments

The most common reason for conflicting feedback is a poorly defined target audience. If half your interviewees love a feature concept and the other half hate it, you likely have two different Ideal Customer Profiles (ICPs).

Review the demographics, firmographics, and daily workflows of your interviewees. Group the feedback by these attributes to see if the contradictions resolve themselves. If you need help narrowing down your audience, review how to define acute ICP pain points for startups to ensure you are targeting a specific, unified group.

Step 2: Build an Evidence Matrix

An evidence matrix helps you systematically evaluate contradictory claims by weighing them against actual past behavior and the context of the interview.

Create a spreadsheet or document with the following columns to evaluate the conflicting statements:

1. The Contradiction: What is the specific disagreement?
2. User A Context (Segment & Role): Who provided the first viewpoint?
3. User B Context (Segment & Role): Who provided the opposing viewpoint?
4. Behavioral Evidence: Did either user provide an example of past behavior that supports their claim, or was it purely hypothetical?
5. Bias Check: Was the question phrased differently? Was the solution pitched before the question was asked?

Hypothetical Example: B2B SaaS Evidence Matrix

Imagine you are building a tool for remote team collaboration. You receive conflicting feedback on whether teams want synchronous video features or asynchronous text updates.

Step 3: Prioritize Past Behavior Over Future Promises

When two users disagree on what they will do, look at what they have done.

If User A says they would never use your product, but they currently spend five hours a week manually performing the task your product automates, their behavior indicates a severe pain point. If User B says they would use your product every day, but they currently take zero steps to solve the problem, their feedback is likely a false positive.

Always weigh demonstrated past behavior heavier than stated opinions or future predictions.

Step 4: Follow Up on Outliers

If you find a contradiction that cannot be explained by segmentation or bias, do not average the feedback. Averaging feedback leads to a mediocre product that serves no one perfectly.

Instead, reach out to the outliers. Ask clarifying questions like:

• "You mentioned you wouldn't use this feature, but you also said this workflow takes you hours. Can you walk me through the exact steps you took last time you did this?"
• "What specific tool did you try before that made you dislike this approach?"

Key Takeaways

• Do not rely on majority votes: Ten aspirational opinions do not outweigh one piece of hard behavioral evidence.
• Context is everything: Contradictions often disappear when you segment your users properly based on their specific roles and workflows.
• Use an evidence matrix: Systematically evaluate conflicting claims by looking for interviewer bias and weighing past behavior against future promises.
• Look for the truth in behavior: What customers do is always more reliable than what they say.