STR.7:11 - SoTA-Echoing

When is experimental learning worth further commitment? Camuffo and colleagues’ 2024 replication and extension studies a scientific approach in four randomized trials involving 759 firms and reports more idea termination with a non-linear pattern of pivots. Adapt the decision-linked hypothesis and termination contribution in sections 4.1 and 4.5. Reject “always build and test” as the default: sufficient existing evidence can justify stopping. The studies concern the reported training interventions and firms, not a guarantee of revenue or proof that every strategic question needs an experiment.

For accountable experimentation, adapt the OECD’s 2024 policy brief and 2025 discussion of policy experimentation. Their attention to authority, capacity, evaluation and later disposition changes section 4.4’s bounded design and section 4.5’s separate continuation. Policy cases inform this adaptation; they do not replace domain participation or assurance methods.

For constructing a purpose-fit test, adapt the question-first move in Ries’s The Lean Startup (2011, Part Two introduction and chapters 5–6) and his MVP guide, alongside current GOV.UK prototyping guidance. Section 4.4 works backwards from the needed observation and varies what must actually function; section 5.3 makes that choice usable. Reducing a final product’s feature list is a weaker default when the remaining features cannot answer the question. A qualified existing result remains cheaper when it suffices. The startup accounts are historical illustrations, not general effectiveness estimates; the government guidance distinguishes prototype interaction from production readiness in its service-design setting. Neither supplies a technical performance or participation result for this experiment. Reopen when the inference needs different fidelity, exposure or support.

For attributing a change to an intervention, adapt the concurrent comparison in Ries’s chapter 7, with NIST’s randomized-design guidance. Section 4.4 makes allocation an actual random procedure, not a label for successive customer groups. The 2023 review by Larsen and colleagues, especially sections 1.2 and 4–6, qualifies inference through the experimental unit, observation horizon, analysis and interference. Its online-experiment Methods need their stated assumptions; the book’s reported successes do not establish general effectiveness. Reopen the design when those conditions no longer support the required effect claim.

C.11.DUA supplies the full-burden comparison against a sufficient answer or cheaper continuation. This preserves a useful experiment when it can change a consequential choice, while avoiding work whose plausible results would not improve the use. Reopen when evidence, exposure or attainable means change that comparison.