Problem Interviews and Assumption Tests

Problem interviews help a team understand how people currently experience a situation. Assumption tests examine a specific uncertain claim that affects a product decision. Used together, they can expose a weak explanation before a team invests heavily in a solution.

Start with the decision you need to make and the uncertainty that blocks it. Ask about recent concrete behavior, then choose a small test that can change the decision. A participant liking a proposed feature is useful feedback, but it is not the same evidence as observing the problem or the proposed solution working.

TL;DR

Quick Example

This complete Python illustration classifies a fictional research-planning record. An assumption and decision need behavioral evidence; an observation still needs an appropriate test before a result is recorded.

All assertions passed, including preservation of an inconclusive result. The script does not evaluate people, analyze real interviews, establish sample sufficiency, or validate a product. No participants were recruited or contacted and no experiment was run.

Core Concepts

Interviews answer contextual questions

GOV.UK’s user-research guidance recommends planning around research questions, obtaining informed consent, and using open, neutral questions. It also distinguishes interview work from trying a prototype, while allowing the methods to be combined deliberately. In-depth interview guidance

Assumptions need a decision boundary

An assumption becomes testable when the team names the relevant audience, circumstances, behavior, and result that would change its next action. “People want this” is too broad to guide a useful experiment.

Evidence types differ

A report of past behavior, direct observation, a prototype task, and an operational metric can each inform a question. Keep their limitations visible rather than forcing every observation into a single validation score.

Turn a broad belief into a useful interview

These are illustrative prompts, not a fixed script. Use the participant’s context and follow the relevant event. Ask for sequence, constraints, alternatives, consequences, and artifacts where appropriate and consented.

Avoid pitching the solution before understanding the situation. A participant may try to be helpful by evaluating your idea instead of describing the work. If concept feedback is a separate research question, introduce that stage clearly and preserve the distinction in the notes.

Recruit around the decision’s scope. Include meaningful differences in role, access, workload, or environment. A convenient group of enthusiastic colleagues may explain their own experience well while failing to represent the people whose workflow you intend to change.

Handle recording, notes, access, retention, and quotations through the approved research process. Collect the minimum useful detail and separate contact information from broadly shared findings.

Design a bounded test of the risky claim

Choose the smallest test that addresses the current uncertainty. A prototype task may reveal whether people understand a workflow; it will not establish operational reliability, purchasing authority, or sustained adoption.

Write the hypothesis, method, inputs, observation window, decision criterion, and key limitations before seeing the results. Include a stop condition and a way to record inconclusive evidence. Otherwise the team can keep changing the question until any result looks encouraging.

Prefer a meaningful behavior over a vague statement of interest when that behavior can be observed appropriately. Do not fabricate commitments or deceive people merely to obtain a stronger signal. Any recruitment, consent, or intervention needs the organization’s approved process.

Consider alternative explanations. A failed task can reflect prototype fidelity, unclear instructions, an inaccessible interface, or a mistaken assumption about the participant’s role. An apparent success can depend on coaching or unrealistic data.

Keep the conclusion proportional. A few interviews can reveal a mechanism or important variation; they do not automatically estimate population prevalence or prove a causal effect. If the decision requires statistical estimation or causal attribution, choose a method and sample design suited to that question.

Use findings to change the next action

Separate notes of what happened from the team’s explanation. Preserve contradictory cases and trace important conclusions back to the relevant evidence without exposing unnecessary personal information.

Connect each finding to a decision: revise the problem framing, narrow the audience, test a different workflow, investigate a dependency, or stop the proposed solution. A report that leaves every roadmap choice unchanged may still contain useful learning, but the team should explain why.

Avoid ranking participants as good or bad users because they disagree with the proposed design. Investigate the circumstances that make their behavior reasonable. A workaround can reveal a constraint the product needs to respect.

Review assumptions over time. A promising prototype outcome can be overturned by deployment, support, reliability, or economic constraints. Keep the evidence record small enough to revisit when those conditions change.

Best Practices

Keep the question visible

Name the decision and uncertainty in the discussion guide and test plan. Use each method for the evidence it can produce.

Preserve observations and disagreement

Record concrete behavior and alternative explanations. Keep inferences traceable and privacy proportionate.

Decide what changes next

Use the result to choose a bounded action, including stopping or collecting different evidence when the finding is inconclusive.

Common Mistakes

Bad: A positive interview validates the product.

Correct: Identify which assumption the evidence addresses and which questions remain.

Bad: Ask whether someone would use the feature.

Correct: Investigate concrete circumstances and behavior, then test the relevant adoption claim appropriately.

Bad: Every experiment must return yes or no.

Correct: Allow inconclusive results and explain what would resolve the remaining uncertainty.

FAQ

How many interviews are enough?

That depends on the question, variation, recruitment, and decision risk. Do not use a universal number as proof of representativeness.

Can interviews and prototype testing share a session?

Yes, with clearly separated questions and stages so concept exposure does not silently change the interpretation of earlier evidence.

Did the example validate a real assumption?

No. It demonstrates a fictional evidence workflow and contains no actual participants or study results.

Related Topics

References