MKT.10:5.1 - A promising invitation with an uncertain useful effect

A provider wants to add guided onboarding to an invitation. The decision concerns another comparable group of 1,000 eligible customers. The useful outcome is verified completion of a specified task within 14 days, not clicking the invitation. Existing commitments remain available under both alternatives, and the provider has checked that the additional help can be delivered without reducing that service.

In a constructed trial, 2,000 eligible customers are randomly assigned equally to the new and current invitations. Assume complete follow-up, no material spillover and the same stated outcome rule. There are 120 useful completions among 1,000 assigned the new invitation and 100 among 1,000 assigned the current invitation. The estimated effect of the assigned invitation is 12% minus 10%, or 2 percentage points. Comparing only people who used the help would answer another question and would lose the original allocation’s simple comparability.

For this illustration, an approximate 95% confidence interval uses the difference plus or minus 1.96 times its standard error. With independent binary outcomes, that standard error is the square root of (0.12 × 0.88 / 1,000 + 0.10 × 0.90 / 1,000). The interval is approximately −0.7 to 4.7 percentage points. The positive point estimate is compatible with both a small loss and a worthwhile gain. It does not establish that the intervention improves outcomes. The interval also depends on the stated trial and estimation conditions; it says nothing about an omitted delayed outcome.

Suppose a qualified prospective account values each additional useful completion at 40 monetary units of contribution after its associated delivery expense, and another campaign for 1,000 customers would require 500 units of additional fixed preparation and help capacity. Assume these flows occur within the decision’s funded period. The point estimate gives 20 additional completions, a contribution of 800 and a net increment of 300. Roughly carrying the effect interval through this simplified account gives about −800 to 1,400, even before uncertainty in the value per completion. A positive expected increment alone is therefore a weak basis for an expensive irreversible expansion.

The decision maker chooses another affordable bounded comparison because the unresolved effect can change the decision. If even the favourable plausible effect could not cover the prospective burden, stopping would be more useful than collecting more data. If customers independently need the help to receive an existing promise, that obligation must be supplied regardless of this marketing experiment.

The next proposed group includes organisations with restrictive data access and would exhaust specialist capacity. It is no longer the same deployment condition. MKT.11 establishes a feasible service variant before its benefit is tested; multiplying the earlier 2% by the new audience would conceal the changed intervention.