When analyzing responses from AI-moderated qualitative interviews with Outset, you have two primary types of report questions: question-by-question analysis and transcript-wide analysis. Each serves a different purpose depending on the scope of your analysis and how your interview guide is structured.
Understanding the distinction between these two approaches will help you choose the right method for generating insights efficiently.
Question-by-question analysis
A question-by-question report question analyzes responses to a single, specific guide question. This approach ensures that insights are drawn only from participant
responses to that exact question, making it ideal for focused inquiries.
When to use question-by-question analysis:
When you are only interested in responses to a single question within the guide.
When you want to analyze responses to different interview guide questions separately rather than synthesizing insights across multiple points in the conversation.
Example:
Guide Question #10: “We’d like your feedback on a new concept. Click on the image displayed to see details on a new product offering. What are your initial thoughts?”
Report Question: “What is the general participant sentiment about the concept introduced?”
Options: Positive, Negative, Neutral
In this case, the report should only consider responses to Guide Question #10 - not any other points in the interview where the participant may have mentioned a concept.
If your guide contains multiple similar questions (e.g., separate guide questions introducing different concepts at question #10, #15, and #20), then you would need separate question-by-question report questions for each.
💡TIP: question-by-question analysis is significantly faster to run vs. transcript-wide analysis, as the AI has to sift through less data to gather insights. If you're short on time while generating reports, this type of report question could help speed up your analysis!
Transcript-wide analysis
A transcript-wide report question scans each entire conversation for relevant responses, rather than limiting the analysis to a specific guide question. This is useful when participants discuss a topic at multiple points in the interview, allowing the AI to synthesize insights across different moments in the conversation.
When to use transcript-wide analysis:
When the insights you seek are spread across multiple questions in the interview.
When you want to capture spontaneous mentions of a topic that might arise outside of a single designated guide question.
When you are interested in a broader synthesis of participant responses across the full transcript.
Example:
Suppose you are exploring how participants plan their meals. The interview guide contains multiple related questions:
Guide Question #4: “Thinking about meals you have prepped recently, how did you decide what to eat?”
Guide Question #5: “For your most recent meal prep experience, did you use any tools or strategies to help with meal planning (e.g., shopping lists, meal planning apps or services)?”
To analyze participant meal-planning behaviors, you don’t want to restrict the report to just one of these questions. Instead, you need to aggregate responses across both (and potentially other mentions throughout the conversation).
Report Question: “How do participants currently do their meal planning today?”
This transcript-wide report question will look for relevant responses anywhere in the transcript, regardless of which guide question prompted the discussion.
Choosing between question-by-question and transcript-wide analysis depends on the nature of your research questions. If you need precise insights tied to specific guide questions, use question-by-question analysis. If you are looking for broader themes emerging across the conversation, transcript-wide analysis is the better choice.
Hope this helps! If you have any further questions, please reach out to our team at [email protected] or via chat.