First, the humbling context. The Open Science Collaboration's landmark 2015 effort re-ran a hundred published psychology studies: roughly a third replicated cleanly, and even successful replications averaged half the original effect size. Celebrated effects (certain primes, ego depletion, power poses) shrank or vanished under scrutiny. Industry data tells the same story from the other side: at experimentation-heavy companies, the overwhelming majority of ideas (including well-theorised behavioural ones) fail their tests, and the wins are usually single-digit percentages, not the lab's dramatic swings. None of this means the psychology is fake. It means effects are conditional: on population, context, stakes, and a hundred unmeasured moderators. The lab found the mechanism. Whether it operates on your buyers, in your flow, at your price point, is an empirical question nobody has answered yet.
So the advanced workflow runs a pipeline: textbook effect → contextual hypothesis → falsifiable test → replicated result → shipped tactic, with most candidates dying en route, as they should. The load-bearing stage is the hypothesis, and it has a grammar: Because [mechanism], [this audience] will [this behaviour] when [this change], measured by [this metric]. 'Add scarcity badges' is not a hypothesis. 'Because genuine availability limits reduce deliberation cost, first-time visitors shown real stock counts on high-demand items will complete checkout at a higher rate, without raising 30-day return rates.' That's testable, and every clause disciplines the design: it names who (boundary conditions), the mechanism (what you'll believe about the world if it works), and the guardrail (what harm would falsify the 'win').
Walk one case end to end. A Groww-style investing app reads the present-bias literature and drafts: because immediate rewards dominate distant ones, showing new SIP creators their projected corpus at retirement (distant) converts worse than showing the first month's auto-debit as a completed streak (immediate). Prediction: streak framing lifts SIP completion; guardrails: no rise in 90-day cancellation or in support tickets mentioning 'didn't understand the commitment' (in a SEBI-regulated category, a conversion win that manufactures misunderstanding is a compliance incident wearing a party hat). Design: one variable (the framing screen only, nothing else moves); a true randomised control; sample size computed in advance for the smallest effect worth acting on; duration covering full salary cycles, because payday is this category's tide. Readout: the metric named in the hypothesis, at the horizon named in the hypothesis. Whatever happens next (ship, kill, or iterate) the team has learned something true about its buyers, which is the asset the test was actually buying.
Two disciplines separate this from ritualised A/B theatre. Falsifiability: specify before launch what result would prove you wrong, and let it. The corrupt alternatives are well documented: HARKing (hypothesising after results are known), segment fishing (slicing the data until some subgroup 'wins'), stopping the test the day it crosses significance. Each converts noise into confident nonsense. And interaction humility: effects tested alone don't add. The twelve-triggers page failed not because any trigger was fake but because persuasion elements compete for attention and compound into perceived pressure: System 1 doesn't sum your effects, it reads the gestalt. Test combinations as combinations, or don't ship them.