Everything Connects, or Nothing Does (Judy's Questions 4–5)

Chapter 2 Episode 4

4. What are you going to do about it? Your approach.

5. What are you really going to do about it? Methods, operationalization, and sample.

You have been building the foundation of knowing through Q1 to Q3; then Q4 and Q5 are where the motivation chain turns from understanding into action. Q4 asks about the high-level idea and hypothesis that support the study’s actionable actions, which is also the answer to Q5.

Q4: Your Approach Must Follow from the Gap

Here’s the tricky part. Remember that you have identified the gap in Q3 and proposed a logically coherent rationale for justifying the gap as credible and important, but you could create multiple different rationales for the same gap, and you don’t know which one is most appropriate. The answer to that question, which is also the single most important requirement for Q4, is that your proposed approach should precisely address the specific gap you identified in Q3. If Q3 concluded that existing approaches fail under condition Y because they assume Z, then Q4 must propose something that addresses that assumption.

Here’s a useful test: if someone asks you why you chose this approach and you cannot point back to a specific limitation from Q3 that it directly addresses, the connection is missing; on the opposite side, if a specific limitation from Q3 does not contribute to the reason for the approach you are proposing, then the limitation should not exist there.

Q5: Make It Concrete Enough to Execute

Q5 asks for a &plan specific enough that a collaborator could execute it without you in the room, not just “we will conduct a user study.” It’s like an interview protocol, but here is a concrete research project proposal with design decisions and considerations of details such as which methodology, which measures, which population or data source, which sample size, which procedures, and on what timeline.

Every methodological choice (and I do mean every single one) should be traceable to the approach in Q4, and the approach should be traceable to the gap in Q3. If you choose a between-subjects experimental design, you should be able to explain why that design serves the RQ you formulated, for instance, why it’s more appropriate than within-subjects. If you choose a specific measurement instrument, you should be able to explain what it captures and why that maps to the variables in your RQ.

Never choose a method or determine a design just because it is familiar or convenient to you and you cannot explain why it serves the question. The paper may still be accepted at a top venue, but I wouldn’t consider it rigorous. As I mentioned before, the paper acceptance threshold at a top-tier venue is way below the standard for me (and my lab).

From Design to Execution

Q4 and Q5 complete the design phase of the motivation chain. At this point, you have a problem (Q1), a comprehensive understanding of the stakeholders and context (Q2), a specific gap in the prior work (Q3), an approach that directly addresses the gap (Q4), and a concrete methodology that can be executed and evaluated (Q5). Before moving to the execution and positioning questions (Q6 through Q10), the next post addresses a foundational practice that is especially critical for HCI research: grounding your work in users’ actual workflows through qualitative evidence.