Stakeholder Communication
Communicating results — null results, pushback, prioritization, experiment ethics, and data privacy.
7 questions
JuniorTheoryVery commonPresenting a finding to a non-technical exec — what goes in the first sentence, and what never does?
Presenting a finding to a non-technical exec — what goes in the first sentence, and what never does?
Lead with the decision, not the method. The first sentence gives the answer to their business question and the action it implies — 'churn rose 3 points; we should fix onboarding.' Method, sample sizes, and caveats come only if asked.
Common mistakes
- ✗Leading with methodology instead of the answer and the action
- ✗Reporting numbers neutrally and refusing to state a recommendation
- ✗Front-loading caveats and decimals before the business point
Follow-up questions
- →How do you respond when the exec asks how you computed the number?
- →How does the first sentence change if the finding is bad news?
JuniorTheoryCommonYou are about 70% sure of a result — how do you say that without hedging into uselessness or overclaiming?
You are about 70% sure of a result — how do you say that without hedging into uselessness or overclaiming?
Give a direction with an explicit confidence and a range, then say what would change it: 'most likely up, ~70% sure, plausibly +2% to +8%.' A quantified band lets the decision-maker weigh the risk; a vague 'it might' only hides it.
Common mistakes
- ✗Stating a 70%-confident result as a firm, range-free fact
- ✗Hedging so heavily that no direction or decision is possible
- ✗Reporting only a p-value instead of a plain-language likelihood
Follow-up questions
- →How do you express uncertainty to someone who dislikes percentages?
- →When is 70% confidence enough to act on, and when isn't it?
MiddleTheoryCommonYour experiment came back flat — how do you present the null result so the team learns, not just re-runs it?
Your experiment came back flat — how do you present the null result so the team learns, not just re-runs it?
A flat result is information, not failure. Give the confidence interval — it bounds how big a true effect could be. If the band excludes anything worth shipping, that's a decision, not a dead end. Separate a true null from an underpowered test.
Common mistakes
- ✗Treating a null as a failed test to be re-run bigger
- ✗Reading non-significance as proof the effect is exactly zero
- ✗Slicing until a subgroup turns significant to salvage a story
Follow-up questions
- →How do you tell a true null from an underpowered test?
- →What CI width would let you confidently 'ship nothing'?
MiddleTheoryCommonA stakeholder wants the raw user-level export 'to check it themselves' — what do you give, refuse, and why?
A stakeholder wants the raw user-level export 'to check it themselves' — what do you give, refuse, and why?
Give an aggregate or masked cut answering their question, plus the query to audit it. Refuse a raw PII dump: it breaks data minimization and access rules and re-identifies people. They usually want a definition or filter without personal data.
Common mistakes
- ✗Sharing raw PII because the requester is an internal colleague
- ✗Refusing all detail instead of offering a privacy-safe cut
- ✗Assuming dropping the name column anonymizes user-level rows
Follow-up questions
- →What makes 'delete the name column' a false sense of anonymity?
- →How do you give enough detail to audit without exposing PII?
MiddleTheoryCommonA manager asks you to 'look at it again' until the number supports the launch — what do you say and write down?
A manager asks you to 'look at it again' until the number supports the launch — what do you say and write down?
Checking for a real error is fair; re-running until the number flips is p-hacking, and I say so politely. I state the result, its uncertainty, and method once in writing. They may still launch — their call — but I won't relabel it to fit that.
Common mistakes
- ✗Re-slicing until a version clears the bar and shipping that one
- ✗Treating any request to re-examine as fraud to escalate
- ✗Handing over a favorable figure with the caveats stripped off
Follow-up questions
- →Why does putting the result and method in writing matter here?
- →Where is the line between a fair re-check and p-hacking?
MiddleTheoryCommonA stakeholder rejects your number because it contradicts their intuition — how do you handle the pushback?
A stakeholder rejects your number because it contradicts their intuition — how do you handle the pushback?
Don't capitulate or dig in. Treat the gap as a hypothesis — show how the number was built and what their intuition rests on. Often each measures a different thing: a definition, date range, or segment. Reconcile; fix a real flaw, explain a mismatch.
Common mistakes
- ✗Changing the number to match a senior person's intuition
- ✗Repeating the figure without revisiting how it was built
- ✗Escalating to authority instead of reconciling definitions
Follow-up questions
- →What if reconciling shows both numbers are right for different cuts?
- →How do you invite pushback without looking unsure of your work?
SeniorTheoryOccasionalYou found something unasked-for that kills a project the VP already announced — how do you land it?
You found something unasked-for that kills a project the VP already announced — how do you land it?
Verify it cold first — a costly finding must survive your hardest scrutiny before it leaves your desk. Bring it privately: lead with the decision, separate certain from inferred, offer options, not a verdict. Leave the VP room to change course.
Common mistakes
- ✗Broadcasting it publicly and blindsiding the VP
- ✗Staying silent because it wasn't your assignment
- ✗Delivering raw analysis with no framing or options
Follow-up questions
- →How do you separate what's certain from what's inferred in the delivery?
- →What if the VP dismisses the finding to save face?
