Moderator guide v0.1

Opening script

“We are testing this report, not you. The data are synthetic. Please say what you are looking at and thinking as you work. I will mostly stay quiet. You may stop at any time, and please avoid mentioning confidential work or unpublished findings.”

Confirm consent to take written notes. Open the frozen HTML report at its top.

Tasks

Start timing each task after reading it. Do not define unfamiliar terms. Neutral prompts are limited to “What are you looking for?”, “What does that mean to you?”, and “What would you do next?” Mark any more specific intervention as assistance.

1. “What scientific comparison is this report estimating, at what population aggregation unit, and with what analysis intent?”
2. “Would it be valid to report one pooled effect magnitude across every workflow here? Please explain your answer.”
3. “What can you conclude about robustness within each evidence lane? Tell me what the report supports and what it does not support.”
4. “How many workflows succeeded, failed, were blocked, or were declared incompatible? What happened to the failed workflow?”
5. Name the first successful universe in the complete ledger. “For this universe, what response window and normalization choice were used?”
6. “Imagine this were your study. What would you do next, and what evidence would you still want before making a scientific claim?”

Closing questions

• “What was hardest to understand?”
• “What, if anything, initially led you toward a wrong interpretation?”
• “What information did you expect but could not find?”
• “Which part would you show a collaborator, and which part would require context?”

Only after all answers are recorded may the moderator explain the intended report contract. Do not solicit feature wish lists until comprehension tasks are complete.